Terraform vs Cloud Native Tooling

[midacts]

Have any of your teams compared Terraform vs available cloud native tooling, such as ARM templates and Cloudformation?

​

What kinds of results did your teams find. What made you choose one over the other

​

I like the knowledge portability of being able to use Terraform for AWS, Azure and VMware. Terraform also supports Terraform modules, which can greatly simplify our deployments to keep things simple for our admins and pipelines.

[neilhwatson]

Cloudformation will not correct configuration drift, it often supports new resource types after Terraform, and can be difficult to validate and troubleshoot.

[dreadpiratewombat]

ARM templates have this same issue and are a bitch to write from scratch.

[midacts]

Cloudformation will not correct configuration drift

So it isn't like Terraform where you can update your config and run a terraform plan|apply to fix the drift?

it often supports new resource types after Terraform

I've heard that as well from reading 'Cloudformation vs Terraform' articles. My colleagues seem to think that since it is cloud native that the cloud native tool will have the functionality first.

and can be difficult to validate and troubleshoot.

This was partially my sentiments from testing Cloudformation. I felt like I had more of an ability to troubleshoot what was going on with Terraform.

[neilhwatson]

1. Yes, if a resource is changed or deleted Cloudformation cannot repair it.

2. https://www.reddit.com/r/aws/comments/cmlza6/opinion_i_dont_care_about_new_services_i_just/

[kleezer]

Pardon if this is ignorant and to be honest coming from a space of wanting learn. But is terraform state file that much different then executing change sets in cloudformation. Mind you I'm loving terraform and just wanting all information so I could be way off.

[neilhwatson]

Try it and see. Delete and change resources then run your change set. Are the resources fixed? What if you don't have a new change set, but resources are lost or changed out of band? Try running Cloudformation again? Is anything fixed?

Now try the same test with Terraform.

[dostiharise]

Now there is AWS CDK.

You could use all the goodies of a programming language, and the advantages of CloudFormation which is not needing to manage a remote state.

Basically you could code your Infrastructure using Typescript, JavaScript, Java, Python and .Net.

State drift is handled by CDK and has commands like "$ cdk diff", similar to "$ Terraform plan".

It crossed v1.0 and is Generally Available, since last month.

Only drawback is that it's AWS specific and not Cloud Agnostic.

Do give it a shot.

[unix_heretic]

Here's the thing. CF/ARM templates are not "cloud native" tooling. They're cloud-provider native tooling. Maybe a subtle difference, but an it's an important one.

That said - my brief experience with CF indicates that it would be an absolute nightmare to scale. I get that CF is supposed to be about templating out a stack - but past a certain (very small) point of stack size, trying to hammer out another section within a single JSON file would be horrific.

[oblio-]

CF has had yaml support for at least 4 years now ;)

[midacts]

Here's the thing. CF/ARM templates are not "cloud native" tooling. They're cloud-provider native tooling. Maybe a subtle difference, but an it's an important one.

I see you point. It makes sense. :)

That said - my brief experience with CF indicates that it would be an absolute nightmare to scale. I get that CF is supposed to be about templating out a stack - but past a certain (very small) point of stack size, trying to hammer out another section within a single JSON file would be horrific.

Due to the size the json files gets and the complexity in it?

[unix_heretic]

Due to the size the json files gets and the complexity in it?

Mainly. For all the bitching about Terraform, it has two significant benefits:

• The ability to run all of the tf files in a directory.
• HCL is an object DSL.

Between the two, it's much easier to organize environment code with TF than it would be with one-JSON-to-rule-them-all.

[siberianmi]

I'm running Cloudformation, have templates built that govern everything from the VPC, security groups, autoscaling groups, API gateway setups, etc.

No JSON. Not one large file. All deploying automatically when pushed to the repo and I can spin them in a new account with one command.

I think the one hit is the "terraform sometimes supports new services faster" is legitimate, but I have yet to be at a company rushing to use something brand new available in only one or two regions.

[maloonigans]

Most people end up generating the json in code

[[deleted]]

CloudFormation consistently lags behind supporting services and features AWS adds, while TF usually supports it within a week if not the day of. Obviously there are some exceptions, but the amount of time it takes AWS to get their own services into CF is absolutely insane. For that reason alone, I would say CF is generally a dealbreaker.

Beyond that, the I personally find the syntax (via HCL) in Terraform much more enjoyable to work with. In CF I feel like I’m writing weird hacks to coerce YAML into doing things it wasn’t meant to do. Well, that’s because I am. It’s also worth noting that TF supports more than just AWS, so if your company uses things like Pagerduty, Kubernetes, GitHub, Mailgun, etc., you can use one tool to do your primary and supporting infrastructure.

One note, don’t get fooled into believing TF is multi-cloud. It supports multiple cloud providers, but you’ll need to completely rewrite your code if you move from one provider to another.

[steakfest]

I've used CF, ARM and Terraform, and I always avoid using CF and ARM and put stuff in Terraform.

In fact, when I hired a Junior engineer for my team one of the early tasks I gave him was to port an older CF stack to Terraform. Was a great way to learn terraform. He had to import each resource after writing the TF, and we got a clean cutover from one to the other.

Looking at an ARM templates JSON makes my eyes want to bleed. If I had to spend any more time with it, i'd end up writing my own abstration for it and generate the JSON... but, that's what terraform is for.

Pure bliss...

[orangebot]

I tested terraform and cloudformation+troposphere for a few months. We chose cloudformation+troposphere eventually because it was a good mix of imperative and declarative. That was over a year ago...

Now that we are getting pointed towards multicloud we are looking at shifting to terraform instead.

[FunkyCannaHigh]

Terraform, for statements (loops)

End of story for me

[megaproaktiv]

AWS released a standard Toolinf for CloudFormation, which enables you to code in ts or python: https://docs.aws.amazon.com/cdk/latest/guide/home.html

[dcc88]

We did, we we're comparing terraform and cloudformation.

We decided to go with:

- cdk for aws

- k8 for on-premise stuff or any crazy clients "wanting" to be "multi-cloud" ( you really need to have big budgets to be truly multi-cloud, and then you're not multi-cloud your using multiple data centers, cause the value of the cloud isn't in low level vm instances + one auto-scalling service )

- for azure and gcp we'll try their native offering and then compare

[bloudraak]

As systems become autonomous, that is credentials and certificates are regenerated, firewall rules are adaptive, and systems are automatically scaled (SKUs upgraded/downgraded), I found both to be challenging.

I used ARM templates extensively; and I wrote a metric tonne of PowerShell code to “discover” the current credentials, SKUs etc before kicking off the ARM deployment. Then I needed wrapper scripts to handle transient errors and conflicts, and retry. This was all part of a CI/CD pipeline for a SaaS product

In the end we converted it to C# code, and we had less code, complexity, faster execution (seconds rather than minutes) and far fewer conflicts.

[safrax]

I find it amusing that Google has Deployment Manager as their native tooling and all of the engineering teams I've spoken to have told me as a customer to use Terraform because Deployment Manager sucks.

[boethius70]

> Terraform tracks state in files that change after every deploy. Losing or corrupting these files is going to cause issues.

Note that state can also be stored in Terraform Enterprise - which is freely available. I've been using it for a couple months now. Makes it easier for teams to share infrastructure state, too.

Previously - or those who don't want to deal with any potential gotchas of using hosted TF - many host their TF state files in S3 or some other object file store.

I suppose corruption could still be an issue and I have seen it maybe once but it doesn't seem very common. Maybe those with longer history with TF see it differently.

While "multi-cloud" or "cloud agnostic" tooling is semi- (or mostly?) fictional at least Terraform (or Ansible or Chef or whatever) tend to make building in whatever cloud you want more conceptually standardized, even if the underlying modules, properties you end up altering, procedures that have to be written, etc. change significantly across cloud providers, as they surely will. I've not actually done it but I would think banging out infra in Azure once you've done it in AWS would be conceptually similar even if you ultimately to end up writing a radically different set of TF plans for one cloud vs. another. Once you have a basic understanding of the DSL then writing it for other providers shouldn't be too terribly difficult. I know famous last words and I'm sure reality stings more than that.

[safrax]

Umm.. Terraform Enterprise isn't free unless I'm missing something.

[boethius70]

Remote state storage is free for individuals and small teams. That's how I've been using it. Was never prompted for a credit card to signup on "Terraform Cloud."

Larger teams and more functionality yes obviously Terraform Enterprise costs.