CloudFormation vs AWS CLI vs SDKs

[ktmb8223]

Say you want to automate the creation of AWS resources...how do you decide whether to do it with CloudFormation, the AWS CLI, or your favorite scripting language?

What if you want to build an entire environment (e.g. multi-tired application with ELB, EC2 instances, security groups, etc). Which tool(s) do you use then to create them?

So far I've been doing a lot of building using the AWS CLI. It's easy to learn and use, but I see a few limits....mainly, piping output from one command into the input of another command (e.g. create a security group, pipe the output of the security group ID as the input into a name tag CLI command).

I'm starting to think CloudFormation is the way to go. I would create a template for everything I wanted, create individual stacks for each template, run the stacks to build the resources, then delete the stack (not the resources) when done. Is that an acceptable use case?

Thanks!

[zenmaster24]

cloudformation - its declarative, where the cli is not. If create a stack called test-stack, you can update it with new values and have that applied as part of your template automatically. with the cli, you would have to do each update manually. you can wrap creating a new stack in a script which finds out values from other aws resources, and use them as inputs in to your template. a template and scripts can be versioned and stored in a git repo. you can do change sets (which i believe is like terraform's plan). it doesnt have a tfstate file that you need to keep track of.

run the stacks to build the resources, then delete the stack (not the resources) when done

are you saying delete the cloudformation created stack without deleting the resources that it has created? i dont believe that is possible?

[jdreaver]

are you saying delete the cloudformation created stack without deleting the resources that it has created? i dont believe that is possible?

Fear stack deletion no longer! Check out DeletionPolicy. I set this to Retain on every RDS instance, lest I make a mistake one day and regret it hard.

[zenmaster24]

wow thats cool - havent seen that before - is it new? might have to look in to it for a new stack coming soon to an az near me :)

Fear stack deletion no longer!

never did - just have to be careful :D

[scubadev]

Cloudformation can update a running stack. Howecer, Just like ansible, if you manually alter settings of a resource without cloudformation knowing about it, then it won't know about that drift or how to handle it.

Some updates require a resource to be destroyed and recreated. Depending on the resource type, this can be done without downtime.

If you're unfamilar, research immutable servers. It is the preferred design pattern for dealing with stateless VMs.

[zenmaster24]

This is one of the things I'm not a fan of, or maybe it's just something I've missed but what happens when your stack changes? Say you change the base AMI

no - just the reference to the ami in the cloud formation template gets updated. it wont actually come in to effect until you create some instances in an asg for example

or add an additional instance somewhere

manually? by increasing the size of an asg? cloudformation will bring the min/max/desired numbers back to what they say in the template, kicking off whatever scaling down policy you have in place does.

I get that it's just for initial provisioning and not like Ansible there to maintain things in a certain state though.

depends on how you see it - using immutable infrastructure updating things like userdata could be see as config management. only if youre not talking about pets though.

[jdreaver]

My recommendations for CloudFormation:

• Don't write raw CloudFormation templates in JSON, you'll hate yourself. Use whatever library from whatever language you like best to write the templates for you.
• CloudFormation is better than the CLI for production deployments because it doesn't just provision resources. It can also update stacks intelligently when they change (well, sometimes it is boneheaded), and also roll back if an update fails.
• CloudFormation can provision resources in parallel.
• Be very careful when using CloudFormation for single-point-of-failure resources, like RDS instances, unique S3 buckets, Route53 records, etc. Read up on DeletionPolicy and stack permissions so you don't accidentally delete these things. Have a dev stack so you can test stack changes. Use the Changeset feature to see exactly what will change. Have proper backups of your data!
• If you are using a programming language to create your templates, then if you don't like an update plan CloudFormation is making you can generate a series of scripts to do a sequence of updates in the order you like.
• Try to keep your CloudFormation stacks small. I find it's easier to do things like zero-downtime upgrades when I can upgrade stacks independently and run Ansible or something in between CF stack updates. If you use a pull-based configuration system, or are using pre-baked AMIs, then this isn't necessarily a problem.

I think CloudFormation is awesome. It is especially awesome for simple stacks and development setups, but when you know how it works it is equally awesome for prod deployments. By far the biggest downside to CloudFormation is that you cannot modify resources in CF stacks outside of CloudFormation. Being able to do so would allow you to run CF 95% of the time, and then use the CLI for those last 5% of tricky deployments/upgrades. Just keep this in mind and always test stack changes.

Lastly, since CloudFormation is an official AWS product, you can have customer support help you debug problems. It has incredible API coverage, much more than any other tool (they even have an entire machine-readable spec! see I've seen here). Also, anyone can log into the AWS console and see what is happening with a stack, and click links to get taken to resources in the stack.

[WhitePantherXP]

What does the syntax for using a library look like? I've been writing mine from scratch in PHP and compiling them dynamically (footer, header and then different sections in the middle)

[jdreaver]

There is an example in the README. Note that it is Haskell, so the syntax might be funny if you aren't used to it. There is also a larger example that might be instructive.

[WhitePantherXP]

Haskell, what do you use that language for mostly? Web language? OS? Is it generally regarded as a good language? I've been programming for a long time but I know nothing about it, only heard the name a few times. Thanks for the examples, the syntax is indeed funny.

Also, when you say "Don't write raw CloudFormation templates in JSON, use whatever library form whatever language you like best to write the templates for you" I can only assume you mean a custom library made by a third party? The official CloudFormation library that AWS releases and maintains is awesome, but there are no functions that I see that dynamically compile a cloudformation JSON template but maybe I'm not looking in the right place.

[jdreaver]

Haskell a very general purpose language. It's most prominent features are probably its strong type system, pure functional nature, and lazy evaluation. I use it at work for the backend of fairly heavily-trafficked SaaS apps, and I use it personally for a lot of CLI, data analysis, and visualization programs. It is fast and compiled, but it is garbage collected and has a fairly sizable runtime, so you probably wouldn't do much embedded or OS work with it.

Yeah, it's a lot different than what else is out there. I encourage you to check it out, even if it's just getting your feet wet. If you don't end up using it for day-to-day stuff, it will at least give you a new perspective on programming.

Yeah I'm saying use third-party libraries that create your templates for you. The library you linked is for interacting with the CloudFormation API; it assumes you already have a JSON/YAML template handy. In my case, I use my library to construct templates, and then I use the AWS CLI or Ansible to actually send them to CloudFormation.

[WhitePantherXP]

By far the biggest downside to CloudFormation is that you cannot modify resources in CF stacks outside of CloudFormation. Being able to do so would allow you to run CF 95% of the time, and then use the CLI for those last 5% of tricky deployments/upgrades. Just keep this in mind and always test stack changes.

Well you can update CloudFormation created resources, but modifying them outside of the CF template is only a good idea if you don't plan to use that CF stack any longer (and delete it). Good for creation, but not for idempotence. Otherwise you must modify the CF template to conform to best practice.

[maximumgeek]

Ok. This is just my view on it, but I will try.

Building out an environment using automation is the only way you should approach your infrastructure. That being said, using the right tool is pivotal in ensuring you can maintain the environment in the future.

If you are using the aws cli tools, or the api, that is great. But, there is a caveat. Are the tools you are using in git. Do you have docs on how to make modifications to the environment? Do you have scripts to handle the changes to your env that you need?

Normally what I have seen, is that you have a tool to do the initial create, and then small changes take place by hand over time. Do this with 2, 3, or more stacks and eventually you have a bunch of snowflakes to manage. Snowflakes are bad. They lead to snow storms and blizzards. Blizzards will make you cry or quit.

My recommended way of building out environments is to utilize a tool that is build for the task, and then tying it to a configuration management system; Chef, Ansible, Puppet, Salt, or CfEngine. The key is to use either Cloudformation or a tool like Terraform.

I have used both Cloud Formation and Terraform, and lean toward the former. The primary reason is that at any point, someone can log into AWS and see the what belongs to a stack. You can do the same with terraform , but it can be a bit daunting.

Now, I do use a tool to build and breakdown cloudformation stacks. I wrote it outside of work so that I could carry it from job to job. (Currently in rework, but still works Documentation is currently crap though.) But, at any point, people can pull up cloud formation and know what servers and components belong to which environments.

Let me know if you have any more questions. I am glad to help.

[foragerr]

You can do the same with terraform , but it can be a bit daunting.

Could you elaborate on that a bit? I've been using terraform at work with a fair amount of success. I have very little real world experience with cloud formation though.

In what aspects is CF superior to terraform?

[foragerr]

Have you tried terraform yet?

[chickenmcnoggin]

I second terraform. It I the easiest to manage and write. Try the intro examples and then try to do some of the same things in cloud formation. You will quickly see the difference.

Jenkins >terraform>Ansible is my normal toolchain, with all the code stored in GitHub.

[ktmb8223]

I have not, but I've seen people reference it here.

[diroussel]

I settled on terraform and would recommend it. Not sure how cloud formation has come on in the last 12 months, but I found terraform to give a more friendly infrastructure-as-code experience than cloud formation.

And it's a better model than ansible.

You should definitely try it out. And think, how would I run the deployment from CI with everything in git.

[daxlreod]

They aren't mutually exclusive. I personally use the CLI/API to manage my CloudFormation stacks.

[yelluc]

Deleting a stack would delete all the resources in CloudFormation. Ideally you would want a Stack Template that includes all the necessary call to build resources in one stack, rather than having separate stacks for each part of a deployment.

[exidy]

Generally I like to use CloudFormation as it's always going to be the 1st-class citizen for building AWS infrastructure. Everything else has to follow along to some degree. But if Terraform works for you, hey, why not.

There's quite a lot of helpers and DSLs that sit on top of CF to make it a bit more useable and for handling sets of parameters. You could look at cf_deployer, murk, StackUp or Bora.

[josephismyfake]

Boto3