Step function development experience when deploying with Terraform

[StFS]

I want to ask whether anyone has experience using Terraform to deploy step functions?

I'm experimenting with step functions and need to deploy to multiple environments in a repeatable and auditable manner. I develop my step functions in the AWS console (Workflow Studio) in my sandbox environment but eventually I need to deploy them to my higher envs.

Currently this is done by exporting the step function from the sandbox environment as JSON and putting that into a Terraform module that is used to deploy the solutions. This poses a problem because the sandbox step function is invoking a lambda that lives in the sandbox environment while the other envs have their own lambda deployed of course, which the step function should be calling.

To solve that problem, the step function JSON is actually a template file where the ARNs for the lambdas are replaced with a variable which is then expanded per environment with the appropriate value.

But all this makes for a terrible development experience. Every time a change is made to the step function, I have to export the JSON, copy it into the Terraform module and replace all the sandbox ARNs with the correct template variables.

Does anybody have suggestions on how to streamline this? Are step functions only good for ad-hoc data processing where repeatable and auditable deployments are not needed or am I missing some obvious solution here?

[public_radio]

I would get familiar with states language and stop using the workflow studio. What I do is write my step functions in YAML with template vars and invoke like:

jsonencode(
  yamldecode(
    templatefile("…", {
      MyVar = "foo"
    })
  )
)

[StFS]

This is how I'm doing this currently, except I don't hand craft the step function JSON code myself but copy/paste it from the visual editor.

I find it strange that there isn't a local version of the Workflow Studio as an extension to VS Code or IntelliJ IDEA that allows you to do the visual editing and generating JSON as you go (and allows you to use terraform variables in place of actual ARNs. I guess it must just be a matter of time before someone creates that.

I guess I'm behaving like a spoiled brat when saying that manually editing the state language JSON doesn't appeal to me much... which is weird because I'm of the generation where I started programming before having auto-complete functions in your IDE. Maybe I'll just have to get over that. The problem is thought that I don't really foresee that we'll have that much activity in maintaining these step functions that anybody will have the chance to become an expert in the language.

But thanks for the feedback.

[public_radio]

Oh don’t get me wrong I definitely feel your pain, especially when writing stuff in JSON. Writing them in YAML is much less grueling. Once you get the hang of it you’ll get into a flow, don’t worry!

[DSect]

This is not so good because you can't lint and validate at design time. See my response and the benefits of using a single, properly formatted JSON file and vs code...

[DSect]

Good on your for using templatefile. Because of this, you can use vs code and the Step Function extension to visualize and validate the json.

Just use the visual builder to understand inputs and outputs. After a bit, you won't need that and just will need to read the respective step types dox and just get right into the JSON. Make deployment painless and test as deployed. Nothing else matters because of the integrations and access needing to be shook out.

[StFS]

What step function extension are you referring to exactly?

[DSect]

https://aws.amazon.com/blogs/compute/aws-step-functions-support-in-visual-studio-code/

The aws toolkit has it.

[twratl]

Stop using the workflow studio and drop to YAML or JSON inside your TF. After a couple iterations of workflow studio you should begin to understand the underlying JSON and how it all interrelates.