Ben Bridts: Infrastructure-as-Code, Getting the Most Out of the CLI, and Other AWS Tips

About Ben

Ben has been using AWS professionally since 2015 and, as an AWS Technologist at Cloudar, works with businesses ranging from start-ups to enterprises. Being part of a Premier AWS Consulting Partner, he provides architectural and operational support and shares his experiences along the way. He is also an AWS Authorized Instructor and gives AWS Classroom Training at The Campus.

He has a broad interest in serverless, automation and enabling builders. Currently, he counts CloudFormation, Lambda, KMS, CloudFront, and Lambda@Edge among his favorite services. Still, he has a soft spot for everything related to operational tasks, like Systems Manager.

You can find him online on his blog, Twitter, LinkedIn, and GitHub.

Transcript

Adam: Hey, everyone. Welcome to AWS FM, a live audio show with guests from around the AWS community. I'm your host, Adam Elmore. And today I'm joined by Ben Bridts. Hi Ben.

Ben: Hi everyone. Thank you for having me.

Adam: Yeah. Thanks for joining. So you're an AWS Community Hero, and I've caught a lot of your talks in the past. I'd love to just start with your story for those on the call that maybe don't know who you are. Just how you got into Cloud, how you got into technology, however you want to take it.

Ben: Yeah, sure. So I started working for the company that I still work at today Cloudar, I think around six years and 10 months ago. Kind of rolled into it. It's a company that's focused purely on AWS. So started there as the first employee and then was started building at clients, learning about everything that was to know about AWS at that point. That was like something you can do on your own. There weren't that many services yet.

Ben: Started doing that at clients boats as a Managed Service provider, but also as consulting. Build a lot of things, make mistakes, learn from them and we basically been doing that. And then was also lucky enough to be able to share those things that we learned along the way, basically from doing them, experimenting with them. The fair dice thing about Cloud is that the cost of failure is so low. So you can very easily look at the new service and then like, "Oh, I wonder what that does or how I can combine those things together." And just try it out. And then if it doesn't work, throw it away and if it does work, you can start implementing that at clients and start working with it.

Adam: So you were new to the Cloud when you started at Cloudar. You were the first employee and got to learn on the job as you went.

Ben: Yeah. So I spend a little bit of time before that working as a developer for a software at a service company. That software for festival management, so artists management, stuff like that. But actually I studied Systems and Network Engineering, so I had more of operational backgrounds. So the step from more the operational focused part of that at Cloudar, building with clouds was a very easy step and a very expected step. The software service thing. We knew both that company. I mean, this is going to be like a temporary thing for a few months and then I'm going to move to something that's more on the operational side of things.

Adam: I saw your talk at CDK Day. You spoke a lot about Infrastructure As Code, your experiences there. It's new to a lot of people. It's not new to you. You've got a long history with Infrastructure As Code. Could you talk just about that and how that's come into your career?

Ben: Yes. So I started basically the first weeks working with Cloudar making these [inaudible 00:03:28] codes. At that point the options were much more limited. I'm not sure when Terraform started, but I assume it already existed, but there was no YAML in CloudFormation. Lambda wasn't [inaudible 00:03:42] yet. It was still a better service. And so we started using Troposphere at that point, which is a vital library on top of CloudFormation. There are some alternatives. They have the same kind of thing, existing Ruby [inaudible 00:04:00] and stuff like that. And started building with that and because if nobody likes to write GC, at least. And that is actually something that is fairly close to what CDK does. It's not as purpose-built for it, but I got to try that out. Figure out better as in it's learn from other people using it, see things that went well with it. And mostly also realized when I made things more complicated by using that extra layer instead of sticking to the actual CloudFormation part underneath.

Adam: Yeah. And that was my next question is, in 2021 now that we do have more options. Do you have a preference in terms of the many different, IAC tool sets out there? Is there something you lean towards today? If you're picking for a new project, you get to choose whatever you want, what do you lean towards?

Ben: Yes. I still think that CloudFormation and then Native CloudFormation. So the YAML or something very close to that should be the default. There are definitely reasons why you might pick something else. There's a reason that CDK is popular, right? It fixes something. So there are some points with CloudFormation and the same thing can be said for Terraform or Pulumi or all the other things that exist. But from operational perspective, if you want to optimize for operations, which I think in a Cloud world, the right thing to do, you can call this more serverlessly or reducing operations or reducing all those kinds of things. CloudFormation is focused or is best for the least amount of operations. Every other solution you take on some extra operational aspects.

Ben: And if you can avoid that, that's a good thing. They do the other things, right. If you need to manage a lot of things that are hybrid, you might decide that extra operational overhead that you get from using Terraform is worth it because you don't have to write a resource provider for every other thing that you're using that's not running inside of AWS.

Adam: Right.

Ben: But you have to be aware that you actually are taking on that overhead and you can't use all those very strict or all the pre-built integrations that CloudFormation has with IAM, with Service Catalog, maybe with StackSets. With organizations all those things are things that you lose. You lose the manage control plane. You have to build a pipeline. You have to store [inaudible 00:06:57]. So for that reason, I am still convinced that CloudFormation is the right default. And that's a place where you start from, and when you've picked something else, you should be very aware of all the extra operational steps and extra complexities that you might be taking on.

Adam: Yeah. So, let's talk about some of those. So thinking first party here. So with the CDK, if we look at someone's decided they're going to use the CDK, what are they signing up for? What other additional operational things are they taking on?

Ben: It's very easy with a layer on top of a CDK, a layer of ClouFormation, to start to over optimize. And the big thing there is, and it's not necessarily something against CDK, but it's something about how CDK is perceived, is that it's doesn't take away the need to learn everything that you're actually owning and running in the Clouds. So if you look from outside and the way that some people try to use CDK is just, oh, I'm going to build some abstractions. And then another team can use those abstractions and they don't need to worry about everything underneath. And that is true. If you do that in a managed service capacity. If you're in AWS and you're building a service or if you're a Managed Service Provider and you're. The same goes for Proton and stuff like that, where you're actually taking all that responsibility away from the users.

Ben: But if you're saying, oh, we're building the abstractions and now you can deploy it to your account and you own that. Then you're actually making a mistake there I think. Then you're taking away the opportunity for people to learn that, what's actually running underneath, and they will have problems later when they either have operational problems, have to debug something production or you want to add something or want to deviate from what you already built. Suddenly they have to learn everything. They have to learn everything, because they own it, right. It's their infrastructure that they're running.

Adam: Yeah. I just had Sheen Brisals from LEGO. He talks about making an individual's engineer more empowered and more getting into the DevOps side, getting into the operational side, not hiding that from them. And that, that ownership is really healthy within a team. And it sounds like what you're saying, echoes all of that.

Ben: Yeah, exactly. That doesn't mean there's no value in that, right? So sometimes you might want to move fast and break things. That sounds a little cliche. So there are ways to leverage that, but you have to be very aware that operational things still exists and it's still [inaudible 00:09:53] organization. And if the person building the abstractions is not going to take ownership of that, then there's going to be some problem somewhere. Or at least some friction. It's not always a problem. There's going to be some friction. The same thing happens if you want to update that abstraction. Because if you don't own the deployment of that abstractions and you have a breaking change there somewhere, then now you have to communicate with all those things. So you're also increasing that complexity there.

Adam: Yeah. Interesting. So I remember like a couple months ago, maybe a few months ago that there was this debate on Twitter for just a couple of days, but it was Infrastructure As Code versus Infrastructure As Config, IAM or whatever. There was people strongly on both sides saying like, "If you're writing imperative programming code, that's dangerous. And here's why." And I didn't think getting on this call with you. I guess thinking of you speaking at CDK Day, I don't know that I expected that maybe you're on that side that. It introduces a lot of dangers that maybe aren't worth it for every team.

Ben: Yeah. And not to become too preachy. I think everything can be set as writing code. We shouldn't be making that distinction. The distinction between configuration or a high-level programming. And to be honest, CloudFormation is not configuration. You're just writing in a DSL in Domain Specific Language. That's still a program. And even if it weren't. The line between what is configuration and what is actually programming is non-existent in lots of cases.

Adam: Yeah. Yeah.

Ben: It's like when becomes a grain of sand then you have a grain of sand, then when do you have a [inaudible 00:11:43] of sand. It's all the same. CDK definitely it solves a real problem.

Adam: Yeah.

Ben: That's I do think that there need to be some warning signs. Somebody needs to be there to say, "Hey, have you thought about this? Are you sure you want to do these things?"

Adam: Yes. So are there other foot guns or things that you would say when it comes to, not the CDK, but just generally speaking Infrastructure As Code. Things that you've seen along the way in your years of working with Infrastructure As Code. Things to avoid.

Ben: Yeah. Don't be too smart is the main thing. It ties a little bit in that [inaudible 00:12:26] things. The one thing that I think is a good idea to think about is that you have somebody else coming to terms, but you don't want to right level two and a half constructs. So for people who are familiar with CDK, you have level one, which is the translation layer directly to CloudFormation.

Adam: Yep.

Ben: You have level two, which are the ones that are usually written by the AWS Service Teams, so that is still within a service. Oh sorry not that it's a service team but the CDK team. Within the service of giving some more ergonomics around that. And then if you're level three, which are written usually by the organization or end user or community. And in that case, it's mostly, or it's at least meant for patterns and putting things together. When it gets dangerous and when you get in a very tricky situation, is if you try to go to your compliance and governance.

Ben: If you try to code that inside of a level three construct, because those things change. Probably you can start you just say, "Oh, we're going to only allow one or two options and then we can set everything else." But there's always going to be exceptions where people need more options or the underlying service will change. And you will have like, oh, the thing that was the best practice a week ago, isn't the best right now. It's not always the best thing to do. And that's definitely a risk and it's a trap that's fairly very easy to fall into. We're going to say, everybody needs to have in our organization, all the buckets need to be encrypted.

Ben: So I'm going to have a level S3 bucket, and then we're setting an encryption in there. But now you have to maintain basically a level two or level one construct yourself. And whereas people are assuming it works at level three and that's true in every language. If you're using CloudFormation modules, it's the same problem. So you don't want to do that. It's that you want to use things that check for compliance. So things like CFN-guard, where you actually go see, oh, you created a bucket, but it's not encrypted, that's failed deployment, right? So it's not deployed. If you're using CDK, you can maybe use, the aspects. You can use an aspect, for example, say, you need to either run CFN-guard externally, or, we go to check your template and you have these and these things, or we're just going to fail the is the [inaudible 00:15:12] step.

Ben: It just gives you a big error saying, "Hey, we will not synthesize this if you have an unencrypted bucket in there." So it doesn't have to be [inaudible 00:15:21] the governance at the last step, when it's already deployed, you can do it's in the pipeline or in CDK. But it shouldn't be, "Oh, use this module that has compliance built in." Because your compliance tools will change. You will have exceptions or the underlying resource will change. And then now you have to update all of those things. And again, you get that communication ownership, all those things become problems again.

Adam: Yeah. So I'm glad you said this, because I think I had an item in my Trello board. I put a lot of ideas. Things I think of that I want to make at some point. And one of them was in the U.S. we have HIPAA, which is all of the healthcare software has to be HIPAA compliant. Right. I thought like, "Oh, a library of constructs that are just HIPAA compliant versions of all the different constructs that people." You get a lot of requests as a freelancer to build something out and make sure it's HIPAA compliant. But make all these constructs that are HIPAA compliant. Hearing you say this, I think you've just allowed me to remove that from my list. It sounds like that would fall into that pattern of baking those things into a level two.

Ben: Yes. And although I don't want to discourage you of that. Right. There are different ways.

Adam: [crosstalk 00:16:35] I need to thin out some of my ideas then. So this is great.

Ben: I just want to say, there's still value in making actual patterns. But you should be aware that you actually making a better and that you can give a name. I think the CDK patterns are pretty good at that, but it's not just like, "Oh, this is optimized API gateway." But it's like, "Oh, it's SNS and Lambda." It's something else tied together. There's like a name like [crosstalk 00:17:02].

Adam: Yeah, this is a common pattern. You both these services together. And here's one way to do that. Yeah.

Ben: And if you don't need that better, but you do need those services, then you do it's on your own with the level two or level one constructions right. You're not going to be, "Oh, I'm going to change it better and mix a Lambda [inaudible 00:17:20] there." That's not going to happen.

Adam: I'd be curious your thoughts on the future of Infrastructure As Code and specifically, they just, yesterday I think made this Proton Inc. made Serverless Cloud public. So I've signed up and played with that. And it has an interesting take where they're interpreting your application code and deriving infrastructure from that application. So it's even further removing you from this operational concern. And I'd just be curious whether you have thoughts on Serverless Cloud or not. If you just have thoughts on the direction of Infrastructure As Code.

Ben: I do think we will see that kind of thing more. I don't have any projects out of my head, but I know people in CDK are looking at doing things like that. I think Pulumi has things like that as well. I'm not a big fan of it, mostly about that same ownership thing again. If you know what it's actually creating, then it's probably fine. If it's just going to do a lot of magic, you still own the things running AWS account rights. So it's a tricky thing there. I do think there's a reason it's very popular and it's also not a new thing. If you look at, I think it's called [inaudible 00:18:47] can actually, it's not exactly the same, but you basically annotate your codes, just like you within Flash.

Ben: You can say, this is my hello [inaudible 00:18:57] route and things like that. Then they'll set up your API Gateway for you and stuff like that. I do think we'll see more of that. Again there, I think the ownership aspect is very important and knowing what actually happens, because otherwise if you want something where it just like throw your go through and it sets up everything that should be a managed service. That shouldn't be something that you have to build together yourself.

Adam: Right. [crosstalk 00:19:28] For ultimately what's going on in your account. Yeah. That makes sense. So, any other thoughts on Infrastructure As Code, I'm going to shift away from that, but if you had anything else you want to share on that, I didn't want to move too quick.

Ben: I do think we will still see more improvements there. I'm very interested in seeing. There's still some pain points, especially with the AWS, the multi account things. Everything that relates to IAM and to things like that, and integrations. Yeah. It's always tricky. And the recently released Cloud Control API. I'm not sure if you saw that.

Adam: Yeah. Yeah. It was a big release day, I think. Right. There is several things that was kind of...

Ben: Yeah, it was a very busy day.

Adam: Yeah.

Ben: I think that will enable some things, especially for third party providers. I think that's a good thing. I think not having to reinvent the wheel and getting part of the things that CloudFormation does now, having that available for other services or other tools is definitely a good thing. That's something that we can all benefit from.

Adam: Yeah.

Ben: So I'm very interested in that cross there where you can have, basically things moving faster and especially because this will also hopefully make it an even higher priority for the service teams to have a very thought out resource schema there. So, yeah. I don't know. The way that CloudFormation works is they have to define that provider, provide a schema for that, it has to match their APIs.

Ben: Not all schemas that get released are the best. There are some things where they could get some improvements. Sometimes it's because the team itself doesn't really have that much insight in how people are using it, within CloudFormation. Sometimes it's because the API itself is not built for all those things that it needs to do. And I hope that with more providers using those schemas and a resource or more tools using those providers, there will actually be a more of a focus on we don't only need to get the API right. Which is a thing that now Terraform implementing and other tools use. We also need to get that schema right before we even start writing our API codes. Just having that top of mind when the service gets created and when the PRFAQs gets through written. It's hopefully a result of this release.

Adam: I had somebody I'm trying to keep tabs on Twitter and I did have somebody call out. This is an important note here that Serverless Cloud does run in their own accounts, not in the user's account. So if I misrepresented that. So it's taking on that responsibility in their case. Just making a more accessible platform. It's just different from anything that's out there right now. It's interesting to follow that.

Ben: That's very cool. It's kind of a Serverless Heroku in that regard.

Adam: Yeah. Yeah. Super interesting. So you've written and you've done a talk on CLI tips. You don't have to recap all of the CLI tips, but if there's anything since then, or any really key elements of that list that you wanted to just call out.

Ben: Yes. I think my favorite one is... But let me rephrase it. Have you ever done the --debug? If you run the CLI, if something that's broken?

Adam: Yeah I saw that in your tips, I've never done it. No.

Ben: So if you do that, if you do, see Command Reference Debug, you get the full [inaudible 00:23:21] debug output of everything that's happening inside of the CLI, which is a lot. It's just three screens of scrolling. And the thing that you actually interesting in it is, which is the API that gets called PHP request going out, PHP requests going in. It's hidden in there somewhere, but you can't see it, but you have to scroll-

Adam: [crosstalk 00:23:44] Signing code and stuff. Is there a ton of output from that kind of stuff?

Ben: Yeah. And the CLI is basically three layers. You have the CLI libraries, you have [inaudible 00:23:56] which is the Python as a gain. And you both the core, which the data layer of there is [inaudible 00:24:07]. So you see all the things like that it's doing in there. It's like hooks and events and stuff like that. So it's terrible to use. Try it out just to see it. And then once you've seen that you shoot enable the history on your CLI. So there's the configuration setting that you have to do. You have to do it per profile. It's called, I think it's called CLI_history. So if you do AWS configures, CLI_history enable something like that.

Ben: It's Google-able. It actually gets access to the history commands where it can do a history show to see the last command or see the [crosstalk 00:24:50] and that will actually give only the information that you want. So it will show you, this is what you entered in the command line. So you can exclude anything that you best shenanigans. It will show, this is the thing that I constructed in Biton as this is the object I wanted to set. It will show you, this is the HTP request that I send out. And then the reverse, HB response, [inaudible 00:25:13]. And then this is the output that I gave. Which is a lot easier to use and a lot better, or lots of more useful than actually having that fully book outputs where you, it's hard to find something in.

Adam: Yeah. Yeah. I'm definitely after the call, I'm going to try to debug though, just to see, I want to see this mess. What's going on there. So you work at Cloudar. We talked about the beginning? Could you tell me a little bit, just about your role there at Cloudar. You've been there? You were the first employees, you said?

Ben: Yes. So I had the luxury of being able to define my own role a little bit. I'm basically doing three things. We have a Managed Service part of our business. So some of my time is spent helping there. I'm doing some on-call shifts. I'm also looking at how do we improve that across the board. We run the AWS Landing Zone, for example, to provide a standardized set up in every account.

Ben: Sometimes it needs a little tweaking. Sometimes there are things that we might want to get into every account that's not part of the official, AWS Landing Zone. It's like one third of my job, is the helping out there. I have one third of my time that I spent doing consulting. So I'm two days a week at one client that I'm at long term, but I'm also doing some smaller projects in between.

Adam: Yeah.

Ben: Sometimes it's a little bit into that my service time. So it's a little bit in a third part that I'm going to mention now, which is I'm doing these things like this. So I'm writing blog posts, speaking at events, appearing on podcasts on Twitter spaces. So I also get some time to do that, and actually not only learning internally and write documentation internally, but also share those things externally and [crosstalk 00:27:23].

Adam: Yeah. So I would imagine that helps that you're still doing the consulting too. That you're plugged into real customer's problems and sorting through that stuff. That keeps you just applying what you're learning in the space and what you're sharing with others.

Ben: I really like that model for the Managed Service part, where it's very much like a across, where we're not... And I'm not saying this to speak ill of older companies. But we're not coming from the main service part. It's like a company that has been doing Managed Services for like 40 years in all different kinds of firms. We were coming from, oh, we're doing AWS and we're very good at that. So let's take that knowledge and package that as a way to help customers that don't want to have full ownership of their infrastructure, right. That want to have a partner that can help them that knows the ins and outs of those things, where they get focused on, for example, writing codes or what the application actually does. Things like that.

Adam: All right, we're going to take a break here and play a game. This is new to the show and I need to workshop the name, but for now the name is AWS or Amazon. So we all know Amazon Web Services, we've got over 200 services and they all are prefixed with either Amazon or AWS. AWS is notoriously picky about getting that right. And we're going to play a game here. I'm going to read some services to you and you'll just guess, or I guess you're probably going to know the answers, whether it's AWS or Amazon. All right. Are you ready?

Ben: Yes, let's go.

Adam: All right. Lightsail.

Ben: Amazon.

Adam: Aurora.

Ben: Amazon.

Adam: Artifact.

Ben: AWS.

Adam: Outpost.

Ben: Oh, that's AWS.

Adam: EC2.

Ben: That's a hard one. Amazon.

Adam: SageMaker.

Ben: Amazon.

Adam: Look at you. Route 53.

Ben: Amazon.

Adam: CodePipeline.

Ben: AWS.

Adam: DataSync.

Ben: AWS.

Adam: Lake Formation.

Ben: I think Amazon.

Adam: First wrong. Kinesis.

Ben: AWS.

Adam: Oh! You missed the last two, but wow. That was way more impressive than I expected. So let's see. I'm going to Tally that because why not? 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. I think you got 10, right.

Ben: [crosstalk 00:30:23].

Adam: Yeah. Oh yeah. In a minute. I mean, I'm going to play it with other people. We'll see... You're the first, so we don't know how you stand. You're number one on the leaderboard for now. Exactly. That was super impressive. So are you going to re:Invent?

Ben: Yes. Well, knock on wood, still. Travel is not the easiest thing this year.

Adam: Internationally, especially. Yeah.

Ben: Yeah. I'm not going to bore you with all the details, but yes. Things are booked now. So booking still. Everything is at this point we're allowed in. We're allowed out again. So it looks like it will all go well, but it has been a kind of an ordeal compared to two years ago where it's easier to get there.

Adam: And do you have any predictions or AWS wishlist items you hope are fulfilled at re:Invent?

Ben: Yes. Although it's hard. Three years ago, four years ago, it was easier to have accurate predictions because the thing that you had to do is look at what's one thing that I really, really need or really want, that's not in the portfolio yet. So I remember CodesBuild coming out, Codes Pipeline and Codes Commit, I think were released earlier. And there was nothing to do to actual build for those things were easy to break the same thing with EKS. It's like, everybody wants that. They have to release it this year. At this point, the things that don't exist yet are limited.

Adam: That's the problem that they have to solve. [crosstalk 00:32:12]

Ben: But I do expect, definitely some improvements in all the things.

Adam: Yeah. That's a good prediction. I'd say you're probably going to get that.

Ben: Definitely just see if I can figure out one big thing. The other problem is being an AWS hero, I have spoken with some service teams, so I can't [crosstalk 00:32:35] that direction.

Adam: Yeah.

Ben: But I know things that people are working on. So I definitely have to steer clear of those because otherwise I will get in trouble. So I wouldn't say these are re:Invent announcements, but more Graviton things is definitely a safe bet. [inaudible 00:32:57] Fargate, right. So we can do Fargates. We can do Lambda. We can do EC2. RAS. I think at some point we will get everything in Graviton. Graviton3 would be nice, but I think that's more something for next re:Invent. It feels a little bit... People are still migrating to Graviton2 so I don't we'll see Graviton3.

Ben: There are a lot of things that are promised from last re:Invent that I think will be announced like the newer Mac Minis with the M1 Macs. I think is probably something for this re:Invent. I think Panorama is still not here yet. So that might be something for this re:Invent. There are probably some other that are not there are around that. Serverless Aurora, the version two of it. I think it's still not here yet. So I think that's also going to be a re:Invent announcements.

Adam: Before I get you out of here, it's been so good to have you. I wonder, do you have any hot takes? I feel like you might. Anything in the industry at all that you'd want to opine on.

Ben: I just got to say like the hills I will die on, so everything is programming. So if you're right. [crosstalk 00:34:19] Anything that's even just relating to any kind of system you're a programmer.

Adam: Yeah. [crosstalk 00:34:27] Yeah.

Ben: I don't think it should be [inaudible 00:34:29]. I think that's a good one to go out with them.

Adam: Yeah. I love it. Thank you so much, Ben, for joining and thank you to everyone that came on to the live show. Man, this has been so great. I've admired you from afar. It's nice to see your face, get to know you a little bit better.

Ben: Yeah. Thanks for having me. I had a lot of fun. I think you're doing a great series of podcast episodes. Every one that I heard was like top notch, so...

Adam: Oh, thank you. Yeah. We've had some, some great folks from the community. And tomorrow, if you want to join us in that, we've got the early slot. So it's the more European and international friendly. We've got Luke van [inaudible 00:35:12]. He's coming on. We're going to talk. It'll be about certs and all kinds of stuff. We're part of an elite 12 AWS cert club that we will talk about that doesn't have a discord or anything about it [inaudible 00:35:24].

Ben: Yeah. Luke is great. You should definitely tell him I said that testing in the CDK is one of the downsides not of the upsides. Just [crosstalk 00:35:35].

Adam: Another hot take. Thank you so much, Ben. I'm going to close out the space. Thanks to everyone that joined.

Ben: Thank you. Bye.