The Modern .NET Show

S07E15 - Deploying With Confidence: Sam Gomez on Terraform, Bicep, and Infrastructure as Code

Embedded Player

S07E15 - Deploying With Confidence: Sam Gomez on Terraform, Bicep, and Infrastructure as Code
The Modern .NET Show

S07E15 - Deploying With Confidence: Sam Gomez on Terraform, Bicep, and Infrastructure as Code

Supporting The Show

If this episode was interesting or useful to you, please consider supporting the show with one of the above options.

Episode Summary

The episode discusses Infrastructure as Code (IaC), focusing on Terraform, but also covering Bicep and ARM. Sam Gomez, a Microsoft MVP, shares his expertise on IaC tools, which enable developers to manage and provision infrastructure through code. He explains that IaC has become a crucial aspect of modern software development, allowing teams to automate the provisioning and management of resources such as servers, databases, and networks.

Sam highlights Terraform as a popular choice for IaC, citing its flexibility and ability to support multiple cloud providers, including AWS, Azure, and Google Cloud. He also mentions Bicep, a newer language that is gaining traction, offering a more concise and efficient way of defining infrastructure configurations. Additionally, ARM (Azure Resource Manager) is discussed as another important tool for managing infrastructure in the Azure ecosystem.

The conversation touches on the benefits of using IaC tools, including increased efficiency, reduced errors, and improved collaboration among team members. Sam also discusses the importance of understanding the underlying infrastructure, enabling developers to make informed decisions about resource provisioning and management. He emphasizes that IaC is not just about automation but also about reducing manual overhead and improving overall developer productivity.

Sam shares his experience as an MVP and conferences, where he helps teams adopt IaC tools and provides guidance on best practices for implementation. He highlights the value of online communities and resources, such as GitHub repositories and documentation, which provide a wealth of information and support for Terraform users. The conversation also mentions the importance of staying up-to-date with the latest developments in the IaC space, including new features and improvements to existing tools.

Episode Transcription

If you talk to people that have been doing Terraform for many years, they’re going to tell you that, “Terraform is the law and Terraform is the way to go.” But like you said, there’s different tools, I would say, or languages that you can use for infrastructure as code. And it really depends what you want to do, what your developers are used to or are comfortable with and what works with your organization as it should be with any tool in software development. You got to grab the one or use the one that is more appropriate for your use case, your scenario, your organization.

- Samuel Gomez

Welcome friends to The Modern .NET Show; the premier .NET podcast, focusing entirely on the knowledge, tools, and frameworks that all .NET developers should have in their toolbox. We are the go-to podcast for .NET developers worldwide, and I am your host: Jamie “GaProgMan” Taylor.

In this episode, Sam Gomez joined us to talk about Infrastructure as Code, why you might want to think about using something like Terraform and Bicep, and how they can help you to automate your deployments to the public cloud.

Sam also talks about best practises for CI/CD and ways to test your Infrastructure as Code ahead of running it–something that we’ve all felt the pain of in the past, I’m sure.

Terraform has what’s called validation for your parameters. So like I said, you can set up a validation that says, “the only values for the SKU for a SQL server are basic,” for example. And if somebody tries to give a different value to that particular parameter, it’ll stop automatically and say, “okay, this validation has failed. You know, the value allowed is this one.” You can do the same thing with Bicep. So that’s another added layer of protection against making these kind of mistakes and adding or configuring the wrong values in your deployments.

- Samuel Gomez

Anyway, without further ado, let’s sit back, open up a terminal, type in dotnet new podcast and we’ll dive into the core of Modern .NET.

Jamie : [0:00] So, Sam, welcome to the show. We’ve been connected for a little while through Taylor’s “Dads in Tech, " which I guess folks should definitely check out. And I always appreciate people coming onto the show and spending time with us. So thank you very much for joining us today.

Sam : [0:18] Thank you so much for having me. I appreciate it.

Jamie : [0:21] No worries, no worries, no worries. I’m always really appreciative of people coming on the show and talking about things that they’re interested in, things that they’re experts in, things that they can share with the community. So then you know gives everyone a chance to learn something new. So there we go .

Sam : [0:40] Yeah, not a problem.

Sam : [0:41] I wonder before we start any kind of discussion on what we’re going to talk about, which spoiler alert: infrastructure is called. I wonder, could you give the listeners a bit of an intro to yourself?

Sam : [0:55] Sure. My name is Sam Gomez. I am a client partner currently at a software consulting company called Geneca. I’ve been doing software development for over 15 years now. Full stack developer. I’ve been working as a consultant for the past six years or so. So I’ve worked with lots of technologies, a mix of technologies, .NET, Java, definitely .NET and Microsoft Stack is my preferred technology. But like I said, I’ve had a little bit of exposure to everything.

Jamie : [1:30] Nice, nice. I feel like that’s kind of the best way to be, in my opinion, is to have that experience with lots of different technologies. And even if it’s just, “hey, you know, I’ve seen a Java code base, I am a .NET dev, and I didn’t really do too much of a deep dive,” you can still learn from looking at other technologies, right?

Sam : [1:51] Yeah, no, I definitely, when I talk to people that are just getting started in technology and software development specifically, I usually recommend consulting as their first job because of that, because you have an exposure not only to different technologies, but to different industries as well. And you might never know that, you know, industry that you don’t really think about as being a technology industry or an industry that, or technology is involved. And you might find something really interesting or an industry that is very interesting from a technology standpoint that way, because you get like that exposure to, like I said, a lot of different things.

Jamie : [2:35] Totally. 100%. I, you know, long-term listeners to the show will know that I work as a contractor. Which is a bit different I think to how it works in the U.S. But essentially the same thing, right? I’m working two, three year projects where I have a defined set of things I need to hit, and I’ve worked worked with companies in loads of different industries like retail, sports, all sorts. And, and you’re right. You get this, this sort of like, I’m afraid of the phrase, but like a baptism of fire of all sorts of different industries and all sorts of different technologies all at once, which is just fantastic.

Sam : [3:22] Uh huh. Yep. Absolutely.

Jamie : [3:25] Amazing. Amazing.

Jamie : [3:26] So we’re going to talk today a little bit about infrastructure as code. Now, I know this as Terraform stuff. I have my own personal opinions on Terraform, which we can go to if you like, but I know that there’s lots of other different ways to do IaC or infrastructure as code. So like, is there one true way or is it like similar to application development where there are thousands of different programming languages and you just pick the one that fits with your problem the best.

Sam : [3:58] Yeah, I think it depends who you ask, right? If you talk to people that have been doing Terraform for many years, they’re going to tell you that, “Terraform is the law and Terraform is the way to go.” But like you said, there’s different tools, I would say, or languages that you can use for infrastructure as code. And it really depends what you want to do, what your developers are used to or are comfortable with and what works with your organization as it should be with any tool in software development. You got to grab the one or use the one that is more appropriate for your use case, your scenario, your organization. So there’s a few out there and especially within the Azure landscape, you have also Arm templates that have been around for a while. And more recently, we also have Bicep that came into the fold for Azure-specific infrastructure as code. But yeah, Terraform, definitely very popular. A lot of people really like it. So those are just three of the tools that are out there.

Jamie : [5:07] Right, right. And am I correct in my thinking that Arm is related to Bicep, like it’s an extension of it or something? Is that correct?

Sam : [5:18] Yes, you got that right. So if anyone has looked at Arm templates, you know that it’s not the easiest format to look at, especially when you have a big deployment, right? You’re looking at basically at a JSON file, and it is easy to get lost. That was my first exposure to infrastructure as code: Arm templates. I actually volunteered to work on that side of the project that I was at the time. And it’s one of those that I sort of regret that volunteering, but at the same time, I’m glad that I got the exposure. But it was definitely, I would say, a little bit of a steep curve because it’s not the easiest thing to look at from a developer perspective. So I’m pretty sure a lot of people feel the same way.

Sam : [6:08] So Microsoft came with Bicep, which is basically a wrapper over Arm templates. So when your Bicep code is compiled, what you get is an Arm template. But it’s definitely a lot easier to look at. It’s easier for somebody that is completely new to look at a Bicep template and have a little bit of an idea of what’s going on, and obviously being able to work with it versus an Arm template.

Jamie : [6:34] Right. Okay. So I write some Bicep stuff. I do a build, I do a compile, I do a transform, I do something. Some magic happens and Arm templates come out the other side, right?

Sam : [6:48] Yeah, that’s exactly right. And you can do the reverse. So there’s a bunch of different tools. So you can decompile an Arm template and get Bicep out of that. You can also, there’s Visual Studio Code extensions that you can use to transform Arm templates into Bicep. And the, you can also use the, I think the Azure portal has a functionality like that as, as well, but there are different ways that you can go from Arm to Bicep and Bicep to, to Arm. If that’s something that you have to do.

Jamie : [7:24] Right. Right. So, let’s say that I’m working on some legacy project that has someArm templates and, I don’t know, [the] project manager says, “we need these upgrading to Bicep!” I can grab one of these tools grab one of these extensions, perhaps do the magic that requires me to convert it; I’ve got Bicep out the other side. Cool, git commit , push, sorted.

Jamie : [7:50] And then maybe I’m working on a different project where they have Bicep templates but for some reason–or Bicep files I guess, I’m not sure, I’ll come back to that in a minute–and for some reason they want Arm templates Instead, It should, in theory, I guess, just be a case of, “hey, extension, application, whatever, here are my Bicep things, give me Arm templates,” at the other end of it.

Sam : [8:12] Yeah.

Sam : [8:12] The only thing that I would say, you know, for the first scenario specifically of moving to Bicep, if you’ve been using Arm templates, it’s going to definitely be an issue or you have to look at the ROI. If you have, especially the big templates with a lot of resources and maybe has some sort of nesting in there, that is definitely something you need to be very careful if it makes sense for you to move to Bicep. That’s one scenario actually where a lot of people have stayed with Arm templates is because Bicep, being a newer tool or a newer language, there’s still certain scenarios where it doesn’t translate exactly what they’re trying to do. So that’s the only caveat that I would say that, sometimes, it might make sense to stay with your Arm templates if you’ve been doing those for a while and your deployments are big, they’re very convoluted. That might be the only option or the only scenario where I’d recommend be very careful about that transition.

Jamie : [9:14] Right, right. And that makes sense right because if it ain’t broke don’t fix it, right?

Jamie : [9:18] So yeah, before I start talking about Terraform–because we’ve talked about Arm and Bicep– just want to get on my soapbox and be a complainy-pants for a moment: From my perspective, when I was told about Terraform, Terraform was meant to be, “you write it once and you can deploy it anywhere!” And in theory that works. Problem is it doesn’t because, you know, you end up writing lots, and lots of cloud specific code; because maybe you are pushing to AWS or maybe you are pushing to Azure, and you have to refer to specific things with specific names; and, you know, there is no, from my perspective anyway, andI’ve been doing it wrong all this time, and I’m happy if I’ve been doing it wrong, because then I can learn to do it differently. But it feels very much like there isn’t a cloud agnostic IaC thing.

Jamie : [10:10] Am I barking up the wrong tree here or have I got my everything wrong?

Sam : [10:16] No, and I think that the key thing, like you said, is when we say that [with] Terraform, you can deploy: write once and deploy anywhere. It’s more so that the language that you’re using, the syntax and some of the things that the commands obviously are going to be exactly the same. But like you said, the specific names of the resources and some of the other, some of the properties, obviously configuration, is still going to be specific for the cloud.

Sam : [10:48] So if you learn the HCL language, which is what Terraform uses, you will be able to deploy to Azure, to Google Cloud. You still have to do some things that are specific for each of those providers, because at the end of the day, it’s not uniform. They all have their own way to doing things. You’re also reaching out to different APIs to create these resources. So all those technically not right ones deploy everywhere is still the most flexible out of some of these tools out there. But yes, still Terraform gives you much more flexibility than Arm or Bicep, which are, as of right now, still Azure specific.

Jamie : [11:31] Right. Okay. And, and all of that makes sense, right? Because at the end of the day… well, actually, before I say anything, let’s talk about what infrastructure as code is. Cause all I’ve done is said, “hey, we’re going to talk about infrastructure as code. Hey, Sam, what’s Arm, what’s Bicep, what’s Terraform?” We didn’t actually say what it is, right?

Sam : [11:49] Yeah. Yeah, that might help for people that are new.

Sam : [11:52] So, you know, not that long ago, and there might still be some organizations out there that when they deploy their resources, their infrastructure, they’re doing it in a manual way, right? We have a person clicking a button, or maybe you’re doing it semi-automated with some sort of script that deploys some resources, especially if you’re doing something like on-prem, which obviously is error-prone. Anytime that there’s human involvement, there’s the capacity to make errors or forget one step, forget to check a box. So that’s where infrastructure as code kind of came in to try to solve some of those issues. So the goal is for your deployments to be automated, repeatable, and consistent.

Sam : [12:37] So automated, we want those deployments to run with our CI/CD pipelines, maybe, or some other automated process without somebody having to click a button. We want them to be consistent because every time that process runs, we want our infrastructure to end up looking exactly the same. Unless, obviously, we make some adjustment to our templates or our files. And we want it to be repeatable because we want to do that, like I said, maybe with every release or maybe with every check-in into our repos. No matter how many times we want to do it, we want it to be repeatable. Two times, 50 times, doesn’t matter. Those are, I would say, some of the main goals of infrastructure as code.

Jamie : [13:23] Right, right. Yeah, because like some of the projects that I’ve worked on, when it’s come to infrastructure as code, it… so what we’re trying to move away from, I guess, is putting in a purchase order and then having someone from ops build a server, and then teaching them how to deploy our applications, right? But some of the projects that I’ve worked on have been , “hey Jamie, you can write all of the infrastructure as code right; because you’re now DevOpsy person as well as being Devy person,” and I’m totally fine with that.

Jamie : [13:59] But some of some of the projects I’ve worked on have been, “hey now that you’ve written the app, now you need to go talk to DevOps and they will write the infrastructure as code for you as well,” right. And it’s felt as though it’s been… and this is I guess this is another complainy-pants moment, maybe this is going to be a complainy-pants episode I don’t know. But it’s felt like it’s like whose responsibility is it right? Is it the devs responsibility? Is it DevOps? Is it infrastructure? Is it Ops? Is it, I suppose ,it doesn’t really matter right; because these tools, these languages make it easier for whomever’s job it is to actually write the bits that are needed right?

Sam : [14:40] Yeah, yeah. That’s exactly right/ And like you said, a lot of the times it’s going to depend on the organization structure, but also the size you know. If you’re working in a small organization, maybe you don’t even have an Ops team, maybe it’s just the devs and, the devs are wearing many hats–which I’m assuming you’ve run into as well. So, that whose responsibility it is, it’s always going to be more an organizational kind of decision. But yes, the end goal is also that anyone with any sort of development experience is going to be able to look at some of these files, some of these templates and be like, “okay, I can make changes to this. I can figure out how this is structured, how to make it work, how to make changes, how to maintain it.”

Sam : [15:29] I’ve seen different ways too where the infrastructure files are kept in a completely separate repo completely different permissions, isolated from the dev team; and some other projects, where the templates are kept in the exact same folder as the application, so that’s more so of a like I said an organizational decision how you want to manage all your code.

Jamie : [15:54] Sure, sure. And because it is effectively code, it is source controllable, right? I mean, literally anything is source controllable if you’re willing to deal with massive binary files. When you start making changes to it, you can actually track those changes like you would any other plain text file.

Sam : [16:13] Yep, absolutely. Yeah, that’s something that you can definitely do, whether you’re doing Azure DevOps or you’re in GitHub. You can check in your files and make sure that you can track all the changes. Obviously, do anything that you would do with any other code that you have checked in in your repos.

Jamie : [16:33] Cool. And the the difference, I guess, is the difference like you said earlier on if you include a human and get them get a… I sound like I’m like a robot or something, but we all know right we’re all human, we all make mistakes. And when you require a person to do some repetitive task, sometimes you know we make mistakes and get it wrong; and so because we’re scripting it with Arm or Bicep or or Terraform, or whatever, it makes it easier for us to then, A)read at a glance what’s going to happen, but also, B) the computer can go away and do it way faster, right?

Sam : [17:12] Yeah, and you can do things like those templates can sort of be self-documented, because you have the ability to add descriptions on things, or add comments, or add text in there that will add a little bit more context about what this parameter is doing, what is it controlling, what is this resource for?

Sam : [17:33] And I don’t know if you or any of the listeners have run into a scenario where you have one person, you have the scenario where it’s one person that knows everything about the infrastructure. That person is great, but there’s no documentation. Nobody knows how this stuff works. If that person leaves or something happens to them, nobody knows how to manage that infrastructure as well or what’s going on. So that’s the other thing that you’re also solving by using these templates: that anyone can look into these templates and see like, “okay, we’re deploying this resource, this is the SQL server, this is the app server. If we need to change something, we need to change it here.” So it’s a little bit easier and it’s more maintainable that way as well.

Jamie : [18:20] Yeah, absolutely. You don’t want to get into a situation where–for listeners who have read The Phoenix Project, right–there’s a character in there called Brent who, it is Brent’s job to deploy the code when it comes to deployment. And at one point in the story, they end up having to, sort of, sequester Brent into a room by himself, and actually run interference to stop people from getting to him whilst he is deploying everything; because, quite literally, as you said, he knows everything else, and so if people get stuck or if people can’t figure something out they go to Brent.

Jamie : [18:56] And that’s that’s the cheery side of it, you know.

Sam : [19:00] Yeah.

Jamie : [19:00] If someone leaves the business or indeed, and you know, I will just say if “the bus factor” comes into play right; then, you know… You don’t want to be in that situation where somebody has left the business, this “somehow” through a good reason, through a bad reason, through whatever reason. And they’ve taken that institutional knowledge with them. And then you’ve got to deploy tomorrow or they’ve left and you did a deploy on Friday, and then you come back on Monday and like the world is… has turned itself over. You don’t want that.

Sam : [19:34] Yep. Yeah, absolutely.

Jamie : [19:37] I like the point that you made about it being self-documenting too, because just like most other code if it’s written well enough you can actually read through it and go, “oh yeah. I get what this is doing. I get that this is doing this thing. I get that this is creating this particular resource, and we’re gonna use its name over here, or its IP address over there.“So I appreciate you for making that that that point there.

Sam : [20:02] Yeah yeah.

Jamie : [20:04] Cool, okay. So this is one of the controversial questions I’m going to have, and for listeners it is one that Sam knows I’m going to ask, is there a reason to use one of the three that we’ve discussed so far over the others? So like we’ve got Arm, we’ve got Bicep, we’ve got Terraform: is there a reason why I might want to use Arm versus Terraform, or Bicep instead of Arm, or like or is it just a case like you said pick the tool that fits the job that that you’re doing at that time

Sam : [20:38] I think the clearest distinction or the easy way to determine if you’re going to need one or the other is whether you have resources in multiple clouds. So immediately going by that criteria, Terraform is going to be the only way to go compared to the other two that we talked about, right? So Terraform is the only one that’s going to deploy to Google Cloud or OpenShift, Azure, AWS. Bicep and Arm are not capable of that at the moment. I’m not saying that Bicep is not going to evolve at one point and do that. But right now, if there is a listener trying to make that decision right now, and they have resources in Azure and Google Cloud, for example, Terraform is going to be the way to go. That’s really the only option that they have.

Sam : [21:30] As far as Bicep versus Arm, which would be the other, I think, more equal comparison. I would definitely recommend for anyone that is just getting started to just go with Bicep, because like I said, it’s more accessible from a development standpoint. You can do things like modularize your deployment. So instead of just being everything in one big file, you can write your SQL server as a module and your app server as a different module, and just put them together in one file that is going to be easier to look, easier to maintain.

Sam : [22:08] You can also use external modules. So there’s people out there that build modules for Bicep and Terraform. So you can just go to the Terraform registry, for example. Or there’s something called Azure Verified Modules for Bicep and Terraform. So these are the Azure verified modules are something that Microsoft is creating these modules, is maintaining these modules. So they have specific standards about how these modules are written. So it’s going to be a lot easier to kind of get started. You don’t have to write from scratch if you want.

Sam : [22:45] And I know I took a little bit of a tangent on the original question, but like I said, the biggest criteria is going to be: where are your resources or where are you trying to deploy? And that’s what’s the main driver between some of these tools.

Jamie : [23:02] Right, right. And that makes sense, right? In the same way that if I said to you, “hey Sam, we’re going to work together on a cross-platform app that we’re going to deploy to iOS and Android, right. It needs to have a single code base.” Uou’re not going to think, “oh well. We’ll write it with Swift,” right? Because we’re deploying it to a place that doesn’t support Swift, right?

Jamie : [23:24] And in the same way if I said to you, “hey Sam, we’re going to work on this thing together. We’re going to write an app, it’s going to be web-based and it’s going to have java, and it’s going to have… it’s going to maybe have a Java backend and maybe a Python microservice, and we’re going to host it exclusively on Azure,” then that means that maybe we don’t have to go with Terraform. We could go with Arm or Bicep, right?

Sam : [23:46] Yep. Yeah. And the other difference or the big difference, obviously, that Terraform has is the concept of a state, which is something that a lot of people working with Terraform love, obviously, because it does give you the ability to determine if there’s what’s called a drift on your infrastructure; which is if you have that user or that client that is, you know, kind of trigger happy and likes to make changes to the infrastructure without following the templates or without making changes to the templates, the state in Terraform will be able to detect that change and let you know, “hey, what we have on our templates doesn’t match what you have out there in the real world. You might want to take a look at that.” That is not something that you have with Bicep at the moment. So that is the other thing that I know a lot of people love about Terraform it does require for you to have some sort of way to manage that state so that it is shared across different developers or across you know the different deployments; but that is, again, and another big difference that I didn’t mention know a few a few minutes ago which is that that concept of of state.

Jamie : [25:09] Okay. So let’s just quickly talk about that then. So there is something somewhere, this Terraform state, we can talk about how we manage that state in a moment, but there is something somewhere that, am I getting this right, that it knows the current layout of what you’ve applied to your cloud provider previously? And then when you come to to deploy again, it is then able to go, “oh cool. You want this new resource. Oh I can see that,” or maybe “oh you’ve taken a resource away, I need to tell you about that.” Is that what it’s doing?

Sam : [25:45] Yep. That’s exactly what it’s doing. So any change that it sees, versus the last time you did a deployment it’s going to let you know about it. Which, again, gives you the ability to say, “wait a minute. That’s not what I want to do,” or, “wait a minute. Who made this change? Who added this resource in the Azure portal or AWS portal and act accordingly. And, you know, figure out what you need to do to get that in sync again.

Jamie : [26:15] Right. Okay.

Jamie : [26:17] So I’m thinking of it a little bit similar to how Entity Framework Core works with this database snapshot, right. When you create a migration, it looks at this snapshot which is a C# representation of what the database previously looked like. And then it’s able to, I guess, speed up that migration process, but also warn you if your snapshot doesn’t look like the database and vice versa. So that’s pretty cool.

Sam : [26:46] Yep.

Jamie : [26:46] All right cool.

Jamie : [26:48] So okay, so I know infrastructure as code in the context of deploying to the cloud. Now I’m going to ask this question, and this may seem like a silly question, but I love to ask the silly questions because they question my assumptions; and I feel like sometimes if I’m making this assumption perhaps someone else is, or perhaps I’m just a wally. I don’t mind being the wally. So is infrastructure as code, things like Terraform on Bicep, things like that, is that just for cloud stuff or can I do IaC for on-prem things?

Sam : [27:24] You can also do on-prem actually. Again, Terraform is a tool that will allow you to do something like that. So if you have VMs that you’re deploying or some other resources that you have on-prem, as long as there is a Terraform provider for that particular resource, you should be able to do on-prem deployments as well. Because, yeah like you said, there might be people out there, it might be listeners that are like, “wait a minute. I’m not in the cloud We have everything on prem, is this something that I’m gonna have to miss out because we we don’t have anything in the cloud?” The answer is definitely no there’s options out there to to do your deployments as well.

Jamie : [28:04] Right, right. So let’s say I’m just going to make some stuff up, right? Hey, listeners, I’m making this up. Nothing of what I say reflects reality in any way, shape or form, right? Just trying to get that out there so no one takes this without context. The context is: I’m going to make some stuff up, right?

Jamie : [28:22] So let’s say that in a magical other world, there is a Terraform provider for, say, Windows VMs that is able to communicate with IIS, right? In this magical pretend world, I could perhaps have a VM that’s on-premises that has Windows 11, say, installed on it or Windows Server installed on it. It has all of the things to do with IIS. I could, in this magical wonderful world that doesn’t exist that I’m making up, I could perhaps install that Terraform provider on that VM perhaps and say, “hey Terraform, go to that VM, spin up a whole bunch of infrastructure around my .NET Framework app, and host it on that machine, expose a reverse proxy via IIS, and tell me that you’re done,” right? Is that the kind of thing you’re saying? Now, imagine that… I’ve had to say, “pretend a whole bunch of stuff exists to make this happen,” right? But is that the kind of thing you’re talking about?

Sam : [29:20] Yeah, I don’t know if it goes that deep. I know that the VM itself, you would be able to spin up that VM and configure some of those resources. I actually don’t know at the top of my head if you’d be able to spin all those other resources that you did mention. And, um, I would assume that would be the case cause you know, it’d be weird if it only takes you half halfway through. But that’s actually something that I haven’t come across yet, because most of our clients are in the cloud. But, yeah, I would assume that that’s something that you would be able to do.

Jamie : [29:58] Cool. Cool. Like I said, magical, wonderful world, silly example. None of it was to be taken seriously. I just, sometimes I work better with examples. And so that’s why I kind of came up with that one. But I guess a different way would be, “hey, I need X number of containers spinning up.” You know, I’m skipping ahead a little bit. Terraform is from HashiCorp, so maybe it works well with Kubernetes. So maybe I can say, “hey, on my on-prem server, I need a Kubernetes cluster. I need this many pods. Each pod needs to be running this container. Let me know when you’re done”, right? That kind of thing? That could probably work, right?

Sam : [30:34] Yup.

Jamie : [30:36] Awesome, awesome Yeah, because genuinely like going into this I was very much in the, “infrastructure is only for the cloud,” right? And no it’s not. This is really cool because, and it makes sense right, because the cloud is just someone else’s computer.

Sam : [30:52] Yeah. And like I said, it’s a lot of people out there that, because of the regulation issues or you know they just don’t want to move their stuff to the cloud. And I definitely don’t, you know, all of these tools, they don’t want to leave those people out in the cold. They want to be able to provide some sort of tool for everyone to be able to do these kind of deployments. Because it’s, you know, it’s really, really good. And it’s sort of life-changing to be able to do some of these deployments, like I said, in an automated fashion.

Jamie : [31:28] Yeah, I agree. I agree. And I really like the idea that if I have the correct providers, and if those providers exist, I could probably do it for my on-prem stuff as well. Because the amount of times, early in my career, where we needed to spin up a new VM and it was, “okay, right, connect to the server using some kind of remote access. Now you’re on the server, go to VMware and install a new VM. Now that you’ve waited 45 minutes for the vm to come up, manually log into the server and change all of these settings. Oh whoops. You forgot one, do it all again. Oh you messed it up. Trash the vm and start it back up because it’s quicker to do that figure out what the problem is. Then deploy the application, then come back out to the the server that’s hosting those VMs and rewire some stuff so that we can do some load…back.” All of that stuff is just it’s way complex and props to people who could do it. I can’t do it.

Sam : [32:26] Yeah. The other thing now that you mentioned that, when you were, you’re saying all that made me think about that. The fact that you can also write your own Terraform providers, for example. So if you have some sort of API and you want to give your customers the ability to manage resources for that API that you are exposing via Terraform, that is something that you have the option to do as well. So a lot of SaaS providers think, you know, Mongo, Datadog, I think Grafana, like all those kind of companies, they expose some of their APIs via Terraform providers so that you can manage those resources in an automated fashion as well.

Sam : [33:11] So again, if you know you’re a big organization, you’re trying to give your clients the ability to manage, I don’t know, create users or accounts or do some that kind of stuff with Terraform, that is an option that you have as well. Obviously, it requires a little bit more work to do that, but Terraform has that flexibility.

Jamie : [33:31] Right, right.

Jamie : [33:32] So that I’m following what you’ve been saying and making sure that I’m not misunderstanding: what we’ve said so far is that Terraform, specifically, is not just for the cloud. It can be if you want. And if you have some kind of service that has an API that you want to expose via Terraform, you can create your own provider. So if I create my own thing, let’s say that I just magically create some kind of software as a service that everyone needs to use. I don’t know, JamieAI or something like that, right? Everybody needs to use JamieAI in their app, right? And then I can write some kind of Terraform service that I can then make sure is deployed to where it needs to go. And then all of my customers that are using JamieAI can then use this. They can add it to their Terraform stuff so that then when they are deploying their app it makes sure to go to jamieai.ai, because it’s got a silly url, and actually make sure that it sets up those resources, and gives them the api keys or gives them the the whatever things they need to then actually finish deploying their entire app.

Jamie : [34:42] Yeah. That’s exactly right.

Jamie : [34:46] That’s really cool. That makes it totally extensible right? Because, as long as you create the thing for it, you can do anything with it. I like that, I like that.

Jamie : [34:56] Okay. So I’ve written my app, I’ve written my IaC, I’ve chosen–doesn’t matter which one I’ve chosen, let’s go agnostic–you know. I’ve chosen Terraform but I could have chosen Arm or Bicep. What do I do next right? I’ve got to apply that somehow. So how do I, how do I run that? Do I, how do I apply it? Like, do I just say to my let’s… okay, well, let’s just dial back. I’m asking too many questions, right? I’ve got some Terraform, or some Bicep, or some infrastructure as code. I am using CI/CD. How do I IaC? How do I get that to run on my server?

Sam : [35:38] So, yeah. So that would be the natural progression. And the great thing is that things like GitHub Actions or Azure DevOps pipelines, they have the capability to run all of these deployments automatically. Obviously, you have to provide some sort of credentials, manage identities in order for these processes to have access to your Azure cloud or whatever the case may be. But you can run them from your CI/CD pipeline.

Sam : [36:11] If you’re using something like GitLab Runners or TeamCity or other tools, you can do that as well. The only thing here, and this is where they might help or might make you decide for one tool or the other, is that not all of these might have support for both Bicep and Terraform, for example. Because, you know, Bicep is a more recent tool. It hasn’t been around as long as Terraform. So there might not be tasks or modules for the deployment of those. So that would be another thing to consider when choosing which tool to use.

Sam : [36:51] But definitely CI/CD pipelines, you can put tasks in there that take your templates from the repo, deploy your infrastructure, and then, like you said, on top of that, you then deploy your applications and anything else that you need.

Jamie : [37:06] Right. Okay. So let’s say I’m using GitHub Actions and I want to go up to Azure. I’m going to host it on Azure. I’ve got my GitHub actions, I’ve got my code. I write this GitHub Action that does something, doesn’t matter what it’s called, but let’s say it’s called “Terraform Apply” or something like that right. I just point it at my Terraform files, and and when it runs, it does… and, okay, I point it at my Terraform files, I give it the role-based access control to be able to do things on my behalf to Azure: I just hit go, and there’s my app. Is that how that works

Sam : [37:41] Yeah, yeah. That’s actually how it works. You can also, you know, if you do something wrong, you can also see the errors there where it will tell you like, “hey, there’s something wrong with your deployment. There’s something wrong with your file.” Or like we said earlier, if there’s something that is out of sync between the state and your infrastructure, kind of, out in the the real world, you can see that in your in your deployment your pipeline as well.

Jamie : [38:10] Right. Okay. So rather than like, if I’ve got a physical script that I’ve got an infrastructure engineer going through: click here, click there, put this name in click there, do this thing. And it’s 25 pages long to set up everything. They get to page seven and things start going wrong. That’s going to take them a little while to get there and ages to figure out; whereas, like, any kind of IaC… we’ve said it already hundreds of times but it’s almost immediate, it’s not super duper [fast], it’s not like immediate , but it’s very, very fast and you get errors out of the back of it. I like that. I like that, yeah.

Jamie : [38:46] Then I could point at someone and say, “this is broken. Why is it broken? How can we fix it?” Instead of getting to the point where we’ve deployed everything and it turns out some resource isn’t deployed in the correct way. I like it

Sam : [38:58] Yeah kind of, to the point that you said that it’s not immediate, you can add things like validations or restrictions that if you want to make sure that nobody accidentally deploys a, you know, a premium SQL Server that is going to cost you, you know, five thousand dollars per hour or whatever. Those are the kind of things that you can also do in these templates.

Sam : [39:20] So Terraform has what’s called validation for your parameters. So like I said, you can set up a validation that says, “the only values for the SKU for a SQL server are basic, " for example. And if somebody tries to give a different value to that particular parameter, it’ll stop automatically and say, “okay, this validation has failed. You know, the value allowed is this one. " You can do the same thing with Bicep. So that’s another added layer of protection against making these kind of mistakes and adding or configuring the wrong values in your deployments.

Jamie : [40:02] Yeah, because I can imagine that would be a very costly mistake to make, right? You do everything, you push it all to the cloud, your app is working, you’ve got a billion users, and suddenly you realize you’ve been paying a million dollars a month for a SQL server.

Sam : [40:16] Oh, yeah. I’m sure that there are some people out there that run into that scenario.

Jamie : [40:22] Oh, yeah. I’m one of them. It wasn’t with Azure. And I will make this story public, I don’t mind. If you all can learn from my mistakes that’s even better. I did some stuff with AWS, clicked the wrong thing, ended up with a thousand dollar charge on my credit card that I wasn’t expecting. And I reached out to them and went, “I’m really sorry. I pushed the wrong button. I’ve created the wrong resource.” And they said, “hey, you made one mistake one time. It’s cool we’ll forgive you for this time.” So that was pretty cool of them.

Jamie : [40:54] But yeah if I’d have done it for, say, work, because that was for a little personal project, if I’d have done it for work and suddenly the boss gets a $100,000 bill dropped on his lap and I’m to blame, I’m not going to be working there for much longer, am I?

Sam : [41:12] Yep, probably not.

Jamie : [41:16] Awesome.

Jamie : [41:16] Okay. So I have got some questions about testing and verifying it, IaC, but I’m just wondering is there a quick description of like: you’ve mentioned it twice, now, this this idea of having state right. So, specifically , with Terraform I can say, “previously on Terraform this happened. This week: the shocking conclusion in Terraform is this,” right? So I need to store that state somewhere. So, I guess, I need some kind of, maybe I use Terraform to create a a bucket of some kind to store that state in, right? Or is that something that is like an add-on somehow? I just tick a box and it starts doing it. Like, how does that work?

Sam : [41:56] Yeah, so you can use anything from, for example, if you’re in Azure, you can create a blob storage container and just store your state file in there. Or you can do it in S3. You can use Terraform Cloud as well. It really depends, again, which platform you use. If you’re already in Azure, then that’s where you want to keep your state file.

Sam : [42:19] And that’s just another concept, or another word that is used for that is the concept of a, “backend in Terraform.” Which, kind of, in the same way that we think about in software development: is that backend has all the information that you are going to need. And you can store it in different places. So you can switch between, you know, when you’re doing things just for fun or just testing things out, y ou use your local backend, which the state file just gets created in whatever directory you work. But once you’re ready to deploy, you need to make sure that everyone has access to that particular state.

Sam : [43:00] So, yeah, you have any sort of a storage solution that you can use will be available for, well, I’m not going to say “any,” but the common ones. Like I said Azure storage, S3, Terraform Cloud. Those are the kind of places where you need to need to put this file and make it available.

Jamie : [43:18] Right, right.

Jamie : [43:18] That, you see, that’s that’s really cool because like it fits with the idea of being cloud agnostic right. And I feel like that was the goal. I don’t want to put words into the mouths of anyone who’s ever worked on the actual implementation of Terraform, but I feel like that was the goal right? And I feel like we’re iterating towards it. And I got upset… I got, you know, what did I call myself? Tt doesn’t matter what I called myself when I climbed on my soap box, and I complained to… “complainy-pants” that’s it! I went into “complainy-pants” mode, and complained about some stuff. But I guess that was coming from a place of not really fully understanding the bigger picture, “of it’s gonna have to be different for every single provider because that’s how it works. Because on AWS it’s called S3, on Azure it’s called blob storage,” you know. You can’t just… it makes perfect sense right?

Sam : [44:14] Yeah. Unfortunately, unless we somehow get all these cloud providers to call everything the same, yeah that “write once, and deploy everywhere,” it’s going to have that little nuance that: the language is the same, but the some specific terms some of the specific values are going to be different.

Jamie : [44:36] Sure. Sure. And that, like you said, it perfectly makes sense though. The reasoning behind it makes perfect sense.

Jamie : [44:45] So okay. Here’s the the the burning question right: I know that if I write some IaC, I can I can push to to GitHub or wherever, and wait for it to run, and wait for it to fail; and that’s about as far as I can test it right. Or can I test? Because we’ve said earlier on that as long as you have the Terraform plugin–I don’t think “plugin” is the right word–but as long as you have a Terraform provider–that was the word–as long as you have a Terraform provider for it, you can run Terraform, kind of, anywhere. So does that mean then that I can have a Terraform provider to let me test my Terraform locally? Because that’s that’s one of the things that always bites me on the butt: is that I type something incorrectly and I don’t know until I’ve pushed it all the way up, it gets to GitHub and it runs, and then it says, “hey Jamie, there’s supposed to be a semicolon here.” Or, “hey duffus,” and is talking to me not you, “hey duffus-Jamie, you’re a duffus. You didn’t put a closing squiggly bracket here. What gives?” right. So how do I… is there a way to test that before I send it up there?

Sam : [45:53] So as far as the format goes, there is… both Bicep and Terraform have ways to check the format of your templates you know do, you know, linting kind of like we do with JavaScript and make sure everything is in the right format. If we are talking about testing as a, like a unit test or an integration test, that is something that you can do with Terraform, actually. I believe there’s something that just came out recently where you can test your deployment and using these test files, you can deploy the resources, make sure that everything looks good. And it will also do the destroy of the resources automatically as well. So more of a traditional testing sense.

Sam : [46:43] But the linting, the format checking, that is something that you can do, and you can add into your pipelines as well. So there’s a specific command that you can run that say, “hey, check my Terraform format. Let’s make sure that it’s correct. " Or, “do a Bicep linting check, make sure that everything, " like I said, “it looks correct as well.

Jamie : [47:06] Right, right. Yeah, because that’s the thing that always gets me is that, like, I get something wrong, and it has to get through. .. because what I also do, maybe I’m doing it in the wrong order. Because, obviously, because I’m a dev more than an infrastructure or DevOps person. All of my GitHub actions or Azure DevOps actions or runners or whatever, they all test my code first. So it does a build, does a test, packages it up, then does the infrastructure. So maybe I should do testing and linting first and then do everything else, you know, every day is a school day.

Sam : [47:41] Yeah. Yeah. And the other thing too, that, that it’s great from, uh, again, if you work with Visual Studio Code, for example, they, there’s extensions for both Bicep and Terraform that will actually tell you like, “hey, you’re missing this parameter,” or, you know, “you’re missing this value or this, ’ you know, “this variable does not exist. " Things of that nature that are a little bit more, you know, more of the obvious side, or I guess more on the critical side. So you have the ability to look at the, or have some sort of warning, even before you check that, or you commit that into your, into your repo. So for everyone out there that is, you know, looking to, to, to work or with, with these templates, I definitely recommend the Bicep extension for Visual Studio Code, if you are in Visual Studio Code. And there’s also a Terraform extension. Like I said, it will help you with a lot of this linting and IntelliSense and all that fun stuff that we love as developers to make sure that we don’t make those kind of mistakes.

Jamie : [48:45] Sure, sure. Well it’s the same thing as IaC, right: if we can have the computer tell us that we’ve done it wrong as fast as possible then, you know, that will help us. That’s why we have linters, that’s why we have autocomplete, that’s why we have these tools, right. I like that, and I guess if folks are in Visual Studio Code, and this is what I really like about Visual Studio Code, you open a file, and Visual Studio Code goes, “hey. I know that kind of file. I have an extension for that. Would you like me to install it?” and you click yes, and it’s there, right.

Sam : [49:18] Yep.

Jamie : [49:18] So you don’t even have to go looking you just go, “yep. Install the extension.

Sam : [49:22] Yep, yep.

Jamie : [49:22] Nice, okay. So you’d recommend the Bicep and Terraform extensions if folks are using VS Code and doing Bicep and Terraform right

Sam : [49:32] Yeah.

Sam : [49:33] Another super fun thing that that I love that the Bicep extension does is: it It has the ability to, if let’s say you already have all your resources in Azure, right? You’re not using any infrastructures code and you want to get started with Bicep, for example. That Bicep extension has functionality that if you take a resource ID from the Azure portal, from that Azure resource, there’s a command that you take that resource ID, you paste it in Visual Studio Code and it generates the Bicep template for you. So I love that functionality, again, for people that are just getting started.

Sam : [50:11] And it also comes with what’s called a Bicep Visualizer. So if you have your Bicep templates and you do right-click, there’s an option for open Bicep Visualizer or something like that. It will give you a graphical representation of the resources that you have on your template. So you can see like, “oh, I have my web app right here. I have my key vault. I have my SQL server, blah, blah,” and make sure that they’re all connected the right way.

Sam : [50:40] I actually ran into something not that long ago where that visualizer showed me that there was a resource that was not connected to the key vault. So I wasn’t storing some of the values in key vault. So that was another really helpful tool. And I thought that was really cool as well. So a few more tips and tricks for people if you decide to go with Bicep.

Jamie : [51:06] I’ve got to say both of those sound, and I’m going to use this word. It’s usually used hyperbolically. I’m probably using it hyperbolically, but “epic” is what they sound like, right. Being able to take a pre-existing set of resources in Azure and create the templates for you, and then be able to–because this is how I would use it, I’d go like, “hey Bicep, I’ve got some stuff in Azure, go get me the templates for it. Cool, now that you’ve got me the templates, let’s see a visual of that so that I can see,” like you were saying, “hey there’s this resource that isn’t wired up to the key vault or isn’t wired up to this that and the other.” That is a really good tip. I love that.

Sam : [51:45] Work smarter not harder. That’s what we say in my house.

Jamie : [51:52] Absolutely, absolutely.

Jamie : [51:54] Okay, so let’s say I’ve never done any IaC things and I want to go and learn all the IaC things where do I go to start right? Is it just a case of googling, “what the heck is IaC?” and following the rabbit hole? Or is there like a like a fundamentals course or something on maybe Microsoft Learn or something that I can take that helps me to learn about it?

Sam : [52:19] Yeah. So Microsoft Learn definitely has a lot of modules about Bicep, and also about the deployment, about the linting some of a lot of these topics that we covered today. If you go to the Terraform website as well, you can also see a lot of tutorials of how to deploy to different clouds, the different commands that Terraform has. I shared links of all of these resources as well for people to just go directly.

Sam : [52:49] But yeah, if you’re familiar with Microsoft Learn and you just type, you know, “Bicep Microsoft Learn, " you’ll see a bunch of modules in there. And like I said, the Terraform documentation is really good as well with those tutorials and sample code that you can just literally copy and paste, and update to work with your deployments as well. So, lots of information out there that’s for, you know, kind of getting started. There’s obviously more courses for more advanced topics. If that’s if people are interested in that as well.

Jamie : [53:24] Nice, nice. And I guess, I don’t know where the url would be, but like if you go to the Terraform homepage… I know that Terraform’s built by Hashicorp and maybe there’s a page on on the Hashicorp website, or maybe the CNCF, there’ll be some resources around those somewhere right?

Sam : [53:44] Yeah, so if you search for Terraform tutorials, for example, there’s the developer.hashicorp.com site. There should be a section in there of tutorials. Like I said, they have tutorials for Google Cloud, AWS, Azure, whatever it is that you’re trying to deploy to. They probably have some sort of exercise for you to get familiar, at least with the basics of Terraform and how to deploy your resources and take it from there.

Jamie : [54:13] Nice. And I know that you’ve sent me over some links anyway l, so I’ll make sure that they are all in the show notes. So if folks are interested they’ll be… however you are listening to this, there will be a button or a link or something that takes you through to the full show notes which will have a list of all of the things that we’ve talked about. So that’s really cool.

Jamie : [54:31] So what about getting in touch with you then Sam? Because I know that you’re an MVP, I’m an MVP, I’m to put your link to your MVP profile in the show notes as well. Like, if somebody’s listening to this and going, “I’d love to pick Sam’s brain for 10 minutes, just for 10 minutes,” like are you on the artist formerly known as Twitter? I know that we’re recording this at a time when there is some stuff going on with X and BlueSky, and there’s some shifting happening. Like, are you on any socials, and if you are, are you okay with people reaching out to you? Like how does that fit with the world of Sam?

Sam : [55:07] Yeah, absolutely. The the one that I’m definitely in the most is LinkedIn: So “Samuel J Gomez” in LinkedIn if you want to reach out there.And I am on X and as well. So on X I’m thesoccerdev, that’s my my handle. And I actually will have to… let me look up at my Bluesky one, because I just started going over there a little bit more, like you said, now that there’s been a shift. But in Bluesky, I’m also thesoccerdev.bsky.social. So same handles, both X and Blue Sky, I guess. So any of those methods, I have no problem, even if it’s longer than 10 minutes. I love helping people. I love teaching them about technology, which is one of the reasons why I do the MVP and conferences and all that: because I love seeing people amazed or relieved that there’s a tool for what they are trying to do. So happy to help anyone I can.

Jamie : [56:12] Awesome. Awesome. Well, I’ll get those links. I’ll put them in the show notes. So if you’re interested in getting in touch with Sam, there’s some links there. Definitely go check those out.

Jamie : [56:20] And I guess all that really remains to say, Sam, is thank you ever so much for being on the show. I say this to everyone, but I genuinely do walk away from all of every single one of these interviews with, like,. loads more information; and I’m happy that I brought up the complainy-pants moment, because then I can be told why I am complaining, which is always… that makes it more easy to digest right? Being told, “yes of course it is like this but here is why.”

Jamie : [56:49] I appreciate you, thank you.

Sam : [56:51] Yeah, thanks . And it helps because there’s, like you said, the the cloud agnostic… I feel like a lot of us think, “okay, I write it once and I deploy it everywhere.” But it’s not quite like that, so it does, the complaining does help sometimes. But you know thank you so much Jamie for having it this was a lot of fun and I hope that people find it helpful.

Jamie : [57:12] Amazing thank you very much

Wrapping Up

Thank you for listening to this episode of The Modern .NET Show with me, Jamie Taylor. Feel free to reach out to me directly, if you’d like to know more about Andys Man Club or, indeed, feel free check out their website. I’d also like to ask that you take some time to do something for you today.

Also, and this is one last thing, please feel free to checkout www.podcasthon.org to discover hundreds of other charities that your favourite podcasters support.

I’d like to thank this episode’s guest for graciously sharing their time, expertise, and knowledge.

Be sure to check out the show notes for a bunch of links to some of the stuff that we covered, and full transcription of the interview. The show notes, as always, can be found at the podcast's website, and there will be a link directly to them in your podcatcher.

And don’t forget to spread the word, leave a rating or review on your podcatcher of choice—head over to dotnetcore.show/review for ways to do that—reach out via our contact page, or join our discord server at dotnetcore.show/discord—all of which are linked in the show notes.

But above all, I hope you have a fantastic rest of your day, and I hope that I’ll see you again, next time for more .NET goodness.

I will see you again real soon. See you later folks.

Music Used In This Episode

This contains some copyright free music during the interstitials. Each of the pieces of music (linked below) were created by YouTube user HoobeZa, and we thank them for making their work free to use. If you liked the music we used, check out links to the pieces below:

Podcasthon 2025

This episode of the podcast contains support for both Podcasthon, Andy’s Man Club, and Capes on the Couch. Please feel free to check out both projects at the links below.

And please remember to check in on your own mental fitness from time to time.

Follow the show

You can find the show on any of these places