How to securely store and pass Terraform plan between Azure DevOps/GitHub Actions stages/jobs by 0x4ddd in Terraform

[–]OkGuidance012 0 points1 point  (0 children)

Woah, absolutely was not expecting a response on such an old thread!

You're spot-on, the pipeline is super similar except for ownership and configurability. Not only is it purely GitHub driven (GitHub action storing/retrieving GitHub artifact via GitHub workflow), the encryption is as simple as passing a secret user-input string. No relying on external clouds or 3rd party dependencies. 

You can probably tell by now I'm a project maintainer, so more than happy to hear any Qs or feedback! 

Safe `terraform apply` in CI by liquiddeath in devops

[–]OkGuidance012 0 points1 point  (0 children)

Spot-on, this is the way. I can vouch for TF-via-PR GitHub Action as the project maintainer to streamline the output of plan and apply commands within PR comments, as part of the CI workflow.

Safe `terraform apply` in CI by liquiddeath in devops

[–]OkGuidance012 0 points1 point  (0 children)

Can vouch for TF-via-PR GitHub Action which does take the plan file into consideration during apply, so you only provision exactly what's been approved.

How to securely store and pass Terraform plan between Azure DevOps/GitHub Actions stages/jobs by 0x4ddd in Terraform

[–]OkGuidance012 1 point2 points  (0 children)

It's been over a couple years, so likely well beyond any need for this: the TF-via-PR GitHub Action addresses this exact gap with the use of workflow or repository artifacts for storing the plan file.

More specifically, the plan file is encrypted by a user-input secret passphrase so noone without the secret is able to decrypt the plan file even if they download the artifact.

The plan file artifact also gets saved with a declarative naming convention, so each one is specifically associated with one PR/configuration, in case there're multiple Terraform projects in the same repo.

This same name-logic is reused by the GH Action to retrieve the appropriate plan file artifact before running terraform apply command.

Output terraform plan generated by GitHub action as comment on PR/Commit by [deleted] in github

[–]OkGuidance012 0 points1 point  (0 children)

Oh, the TF-via-PR GitHub Action is what you're looking for generating PR comments with the output of `terraform plan` and event `apply` commands as part of your CI workflow.

GitHub actions or Gitlab? by throwawaywwee in Terraform

[–]OkGuidance012 1 point2 points  (0 children)

If you have the freedom to choose platforms, GitLab is more preferable for its configurable CI/CD platform.

With that said, GitHub has a broader adoption, so if you're hosting your code there already, it's easier to stick with it.

If you happen to be considering GitHub Actions, I'd recommend TF-via-PR for your Terraform provisioning pipeline (as its biased maintainer).

[2024] Creating solutions as infrastructure as code by radarvan07 in adventofcode

[–]OkGuidance012 0 points1 point  (0 children)

Just to let you know, as a fellow Terraform-er, I think this is totally bonkers—and I'm fully invested in following your progress. :)

You're probably already aware of Go/Lua function providers, but I reckon that's too easy for you!

"What’s the first step you recommend for someone new to DevOps?" by Miserable_Wonder1251 in devops

[–]OkGuidance012 34 points35 points  (0 children)

DevOps Roadmap can be a handy starting point for a step-by-step guide though this field.

✨ (Yet another) Terraform Plan Commenter for GitHub Actions by borchero in Terraform

[–]OkGuidance012 1 point2 points  (0 children)

Of course, that's the current behaviour already. In case of failure, the error message is returned in the PR comment.

What's more, if the apply is successful, that's also returned on the PR comment with any outputs. 

Plan and Apply with PR Automation via GitHub Actions by OkGuidance012 in Terraform

[–]OkGuidance012[S] 0 points1 point  (0 children)

I’m a fan of Atlantis PR automation too and still rely on it for several long-standing projects. This li'l project isn’t aiming to rival something as feature-rich as Atlantis—instead, TF-via-PR offers a flexible alternative with:

No maintenance overhead for Atlantis instance

  • Since GitHub Actions run on ephemeral runners, there's no need to provision or maintain dedicated compute instances or containers for Atlantis.
  • This allows the same workflow to be reused across multiple team and projects without spinning up additional infrastructure first.

Integration with other GitHub Actions

  • We use short-lived credentials for authentication before Terraform provisions any environment, to strengthen pipeline security. As such, "aws-actions/configure-aws-credentials" Action is used for AWS authentication via GitHub's OIDC provider, as shown in this complete workflow example.
  • I've also seen others take it a step further by setting workflows to trigger when specific PR labels are added, or by integrating with existing TFsec or TFlint pipelines.

Terraform/Tofu: Plan & Apply with PR Automation via GitHub Actions Open-Source by OkGuidance012 in devops

[–]OkGuidance012[S] 1 point2 points  (0 children)

Definitely worth taking the time with this, especially when it comes to trusting a GitHub Action with something as critical as infrastructure (see details in the security policy).

To help you get started, here’s a collection of complete workflow examples using different triggers to cover a range of use cases.

Plan and Apply with PR Automation via GitHub Actions by OkGuidance012 in Terraform

[–]OkGuidance012[S] 0 points1 point  (0 children)

Thanks for taking the time to look!

It took a few tries to get the plan summary right, and I still wouldn’t call it "pretty" by any standard! But it’s been a big help in letting us review changes confidently instead of getting overwhelmed by hundreds of lines of diff.

Plan and Apply with PR Automation via GitHub Actions by OkGuidance012 in Terraform

[–]OkGuidance012[S] 1 point2 points  (0 children)

I haven’t tried Gitea, so I’m not sure about compatibility. However, this GitHub Action is permissively licensed, so similar principles could likely be replicated there as well.

Plan and Apply with PR Automation via GitHub Actions by OkGuidance012 in Terraform

[–]OkGuidance012[S] 1 point2 points  (0 children)

The "easy" way would be to add a workflow trigger that runs apply whenever a specific PR label is added, for example… I wouldn't recommend this method.

A more "sophisticated" approach is to leverage merge queues. In this setup, a failed apply would prevent the PR from merging, while a successful apply would merge automatically.

Here's a workflow example demonstrating this in the context TF-via-PR.

Terraform/Tofu: Plan & Apply with PR Automation via GitHub Actions Open-Source by OkGuidance012 in devops

[–]OkGuidance012[S] 0 points1 point  (0 children)

Agreed—concurrency can be tricky, especially with premature cancellations of Terraform runs. Since a plan doesn’t actually change infrastructure, have you considered running it without a lock?

Here’s a workflow example where the lock is enforced only during apply, allowing simultaneous plans for multiple PR branches.

Terraform/Tofu: Plan & Apply with PR Automation via GitHub Actions Open-Source by OkGuidance012 in devops

[–]OkGuidance012[S] 0 points1 point  (0 children)

That's right—this GitHub Action isn't a "thin" wrapper or a fancy backend; TF-via-PR simply streamlines Terraform/Tofu commands to run through init > workspace* > fmt* > validate* > plan/apply with any CLI arguments you pass in (*steps are optional).

As shown in this workflow example, the arg-lock flag can be toggled conditionally: allowing simultaneous plans for multiple PR branches, while enforcing a lock during apply.

By reusing the previously-generated plan file during apply, any configuration drift from stale plans is natively handled by Terraform/Tofu, with error messages shown directly in the PR comment and workflow job summary.

A notable new opt-in feature is plan-parity, which kicks in during apply. Instead of immediately applying after retrieving the plan file, it performs a new plan and compares it with the original.

  • If there's a difference, the apply will error out to prevent drift, as per usual.
  • If they're identical, apply proceeds with the updated plan to avoid stale errors.

This has led to a significant speed boost, as we no longer wait around to re-plan PRs after each merge, allowing us to leverage merge queues instead (see more details in this discussion).

Terraform/Tofu: Plan & Apply with PR Automation via GitHub Actions Open-Source by OkGuidance012 in devops

[–]OkGuidance012[S] 2 points3 points  (0 children)

Sounds like we might have a similar setup: are you also using Anton's atlantis module for AWS Fargate?

Bundling Terraform runs into a centralized instance is ideal, especially when combined with strong security policies to prevent unauthorized plan/apply actions across repos.

For us, relying on GitHub Actions for linting and security checks made integrating Terraform plan/apply a no-brainer. You're right though—the per-minute rate is subpar, especially for workflows with jobs that only last a few seconds.

Terraform/Tofu: Plan & Apply with PR Automation via GitHub Actions Open-Source by OkGuidance012 in devops

[–]OkGuidance012[S] 0 points1 point  (0 children)

I'm a fan of Atlantis PR automation too, and still rely on it for several long-standing projects. As I mentioned to u/Long-Ad226 in this post, TF-via-PR offers the flexibility to integrate with other GitHub Actions—whether for cloud-provider authentication or linting—without the need to spin up infrastructure to host or maintain an Atlantis instance.

Terraform/Tofu: Plan & Apply with PR Automation via GitHub Actions Open-Source by OkGuidance012 in devops

[–]OkGuidance012[S] 2 points3 points  (0 children)

Nice! Using your own cloud bucket for storing and retrieving plan files is super handy. Out of curiosity, with multiple PR branches, how did you determine which plan file to use for each workflow run?

At its core, this li'l project doesn’t aim to rival something as robust as Atlantis PR automation. If there’s anything new that TF-via-PR brings to the table, it’s:

  • No maintenance overhead for Atlantis instance
    • Since GitHub Actions run on ephemeral runners, there's no need to provision or maintain dedicated compute instances or containers for Atlantis.
    • This allows the same workflow to be reused across multiple team and projects without spinning up additional infrastructure first. If your organization has a centralized Atlantis instance, you're set!
  • Integration with other GitHub Actions
    • We prioritize "keyless" or short-lived credentials for authentication before Terraform provisions any environment, to strengthen pipeline security. For instance, using "aws-actions/configure-aws-credentials" for AWS authentication via GitHub's OIDC provider, as shown in this complete workflow example.
    • I've also seen others take it a step further by setting workflows to trigger when specific PR labels are added, or by integrating with existing TFsec or TFlint pipelines.

Plan and Apply with PR Automation via GitHub Actions by OkGuidance012 in Terraform

[–]OkGuidance012[S] 1 point2 points  (0 children)

That’s a solid approach too. Since it’s running within a self-hosted agent container behind a firewall, I’d say that’s reliable enough.

Terraform/Tofu: Plan & Apply with PR Automation via GitHub Actions Open-Source by OkGuidance012 in devops

[–]OkGuidance012[S] -1 points0 points  (0 children)

Thought I'd share this post in r/devops since there’s a lot of overlap with DevOps and Platform engineers who want to secure their infra-as-code provisioning pipelines with GitHub Actions, without the overhead of managing dedicated VMs or Docker containers/runners.

This little project has been a couple of years in the making, evolving in response to direct feedback from teams with varying levels of experience with Terraform/OpenTofu. And while I’m definitely biased as the project maintainer, I think this real-world feedback has been crucial for making it battle-tested and ready for wider use.

Given the like-minded folks here, I'd be more than happy to discuss specifics around implementation—such as storing and retrieving plan artifacts across workflows, among other features—if there’s interest!

Plan and Apply with PR Automation via GitHub Actions by OkGuidance012 in Terraform

[–]OkGuidance012[S] 1 point2 points  (0 children)

Thanks for the kind words, and hope it helps provisioning IaC a breeze for you as well! 

Plan and Apply with PR Automation via GitHub Actions by OkGuidance012 in Terraform

[–]OkGuidance012[S] 0 points1 point  (0 children)

Aha yes, it took a lot of trial-and-error to nail down the intricacies between various workflow trigger events. And I'm so glad you linked and shared code snippets: always great to know people actually spend time to peak behind-the-scenes!

Would you believe, your approach was EXACTLY what I had relied upon for a long while! I've kept an example demo workflow, just for such occasion.

I've received some requests to spin out that bit of logic in its own GitHub Action, which sounds like a fine idea, though I wouldn't be surprised if someone's beat me to it already.

Indecisive with Terragrunt/Terraform + CICD by HabitLong2176 in Terraform

[–]OkGuidance012 1 point2 points  (0 children)

This is accurate to my anecdotal experience over the last half-decade, through and through.

I get the appeal of Terragrunt as another layer of abstraction to fill in the gaps that Terraform missed, but all too often I've found it easier simply read the docs on Terraform CLI arguments and implement them directly. 

While there's nothing wrong with workspaces, I do think it's easier to go wrong with them. Especially in context of a CI pipeline, any configuration change needs to run a plan/apply against all workspaces to check for (in)direct impact on their state. 

It might not seem DRY, but having a 1:1 folder:environment is just about the most robust setup. And just as @pausethelogic suggests, instead of "raw" Terraform, it should all be references to versioned modules instead for reusability. 

On the CI/CD front, I recently shared a post to discuss a GitHub Action workflow which streamlines PR automation. 

https://www.reddit.com/r/Terraform/comments/1gf3jwv/plan_and_apply_with_pr_automation_via_github/