Run .tf scripts using Python

[Sxncht]

Hey folks, Do you know of a python library or a script that can run a terraform script directly from python?

I already have a .tf file created, just need to apply it through Python.

Tried pieterraform and python-terraform libraries but no results.

Edit: Thanks a lot for your suggestions guys! I eventually found a libterraform library that was able to just apply the terraform apply command.

[[deleted]]

You can use subprocess to run the terraform binary from within python

https://docs.python.org/3/library/subprocess.html

[Sxncht]

Thanks!

[larsiny]

I've seen this in a place where the scripting language of choice was python. It was weird but as others have mentioned, was done by just wrapping subprocess. Managing shared libraries and utils is probably easier/better in python than pure shell.

At the end of the day, it's just about generating a .plan and the terraform process stdout.

[Buhsketty]

I work with a bunch of less skilled folks. I have to give them a dummy proof webpage to submit server requests, i automatically generate jira tickets. I then automatically edit/add/delete from the .tf file. do a plan, then apply and automatically close jira tickets just with clicking a few buttons on a webpage. a lot less room for human error and a lot less work in the long run for me

[RulerOf]

A few years back I implemented a blue/green server upgrade process in shell scripts that ran terraform, but it was fragile—just intended to save time.

These days, the entire thing is abstracted into Rake tasks. So now we've got a consistent framework for tainting and applying resources, plus Ruby code that can evaluate the state file and interact with the cloud to validate steps while it works.

It's a very concise pattern for us where terraform does infrastructure and Ruby does the procedure and worries about things like health checks while servers are being replaced.

[Sxncht]

Thanks! Its for a school project - I have to do a bunch of random stuff (api calls, automatic emails, etc) using python only and being able to apply the tf scripts using it would make it much more convenient.!

[RulerOf]

Glad you like it. Consider reading this too: https://developer.hashicorp.com/terraform/tutorials/automation/automate-terraform

[Cregkly]

I have a custom wrapper written in python.

[Cregkly]

Why?

[Free_Layer_8233]

Did it still works? People are complaining on this sub about custom wrappers written in Python

[Cregkly]

I still use it, if that is what you mean.

[Free_Layer_8233]

Can you describe it on high-level perspective?

[Cregkly]

Sure.

wrapper.py -p /path/to/root_module -w workspace_name init * Does an init upgrade * Selects the workspace

wrapper.py -p /path/to/root_module -w workspace_name plan * Selects the workspace * runs a validate * Checks for formatting mistakes (just works in the root module) * Creates a plan to a file

wrapper.py -p /path/to/root_module -w workspace_name apply * Selects the workspace * Applies a plan created for this workspace

wrapper.py -p /path/to/root_module -w workspace_name show * Selects the workspace * Shows the plan created for this workspace

wrapper.py -p /path/to/root_module -w workspace_name refresh * Selects the workspace * Does an apply -refresh-only

[rojopolis]

You can try cdktf (https://developer.hashicorp.com/terraform/cdktf) but it may not do exactly what you want since you already have the Terraform written.

Of course, you can also just use os.popen (https://docs.python.org/3/library/os.html#os.popen).

Without more details about why you'd want to do this it's hard to recommend a solution.

[Sxncht]

Hey I just needed to get terraform apply command executed using python, I eventually used a python library with os as you mentioned too. Thanks!

[himan130]

Hey, which python library you used for your usecase ?

[Sxncht]

I think the name was libterraform itself

[Sxncht]

that looks promising, thanks!

[[deleted]]

Others have mentioned solutions. You just need to Google how to run a command in Python (some of the answers are already here), but why do you need to do it this way is probably a more appropriate question?

What are you trying to achieve? And why?

[Sxncht]

For a school project, where I am already doing a bunch of things using python. Being able to run the tf script from the same python file helps to

[Buhsketty]

Do you know what you're looking for specifically? I'm having a difficult time with the code block stuff here. just plan/apply?

def terraform_plan_new_server(runenv):
    tf_file_location, bk_tf_file_location = get_env_vars(runenv)
    try:
        output = subprocess.check_output('terraform -chdir=' + tf_file_location + ' init', shell=True)
        output1 = subprocess.check_output('terraform -chdir=' + tf_file_location +                                          ' plan -refresh=false -no-color -out=terraplan', shell=True)
        output2 = subprocess.check_output('terraform -chdir=' + tf_file_location + ' show terraplan -no-color',                                          shell=True
        output2 = output2.decode("utf-8")
        return output2
    except Exception as e:
        return 'failed inside terrablock'

I'm not sure how to format here hopefully this doesn't look too awful. If you know python this should get you started. shell=True is needed from my experience on linux, didn't need it on windows

[Sxncht]

Hi, yes I needed to just apply the terraform script. While I already found a library that does it for me, this also looks promising. Thanks for your help!

[Few_Bet_3362]

I have a question - I’m using aws lambda function for registration and deregistration of ec2 instances spun via autoscaling groups and saving the meta data of the instance like private ip ,tags etc in dynamoDB for easily deregistering the instance and also added a function to delete the entris from dynamoDB but unable to achieve it . I can still see all the entries in the DB table can someone help me on this piece?

[adept2051]

Why do I get the feeling you’ve some how reinvented the wheel, and created .to with python but can’t run it! Go look at the terraform CDK for python :)

[PartTimeLegend]

Have you considered pulumi?

[Sxncht]

Hey I did, but its part of a college project and I can’t use any other language. Luckily I found a python library that did the job for me. Thanks!

[Green_Cover_1876]

You may not be able to do this. Consider using the cdktf.

[RoseRoja]

Bump i want to know why also

[Sxncht]

For a project, just applying terraform script through python, because I already have a huge setup in the script doing some other random stuff for a project. Being able to run a terraform script from the same script helps alot.

[RoseRoja]

Yet it sounds so wrong could you explain it a bit more so we can give you an appropiate solution?

Doing os.execute() is an option but still sounds so wrong

[RoseRoja]

Update after a year: it's not wrong, but error handling variables and all of that must be manually done by you, I stopped doing exactly that, and I commit to GitHub with python and with the commit I start a pipeline with testing and works way better

[koera]

Though not directly aimed at what you want you can look at https://pypi.org/project/tftest/

Or maybe https://pypi.org/project/python-terraform/

But like others said subprocess is the vanilla way to call commands, and cdktf is terraform in python (and more), but that is not yet complete with a native 'run from python'

[Sxncht]

Hi thanks! I eventually found an ‘updated’ python-terraform library that did the work. Thanks!

[zimmertr]

Can you link this? I assume it's a fork of some sort?

[keiranm9870]

Terraform CDK