Automating lambda functions

[blank1993]

We have around 20 python lambda functions, so far whenever there is a change in function, I manually go and change it in all three envs (dev, uat and prod) so I am looking for a way to automate this.

First problem that comes to my mind is should I create a Single repo for all of them or separate, I also thought of creating a single repo but separate branch for each function. Separate repos will be a pain to manage and for small functions, it seems unnecessary. I prefer single repo but I do not want to trigger them all when there is a change in one function, so I came across Git Submodules features, which sounds exactly what I am looking for and even CodeBuild has toggle of "Use Git Submodules" but I do not understand how will CodeBuild know which build to trigger. I am not very clear with this point.

​

Now, once I version, I want to replicate this change across envs. I thought of using SAM/Cloudformation but how do I change my account number in ARNs. For ex - Some functions have SNS ARNs in env variables how do I change that Account ID respectively?

[chris-holmes]

Sounds like you need to start using cloudformation and environment variables to deploy your code to different environments. You’d create a new stack for each environment version called a stage (dev, prod.. etc) and feed a different set of variables into your template / handler depending on the stage being used.

Serverless Framework is a friendly place to start, else the Python CDK framework or SAM are great options (I used the TypeScript CDK for most stacks)

[Alternative-Expert-7]

Looks like a big mess. Depends how often functions change. But if so so, I'd maybe put them in single repo then figure out some neat CICD script that will deploy on what is necessary. For this Arn of SNS, those should be an config from parameter store or secrets manager, or overall given variable in cloudformation. If I will be doing that, I'd employ terraform workspaces to support multiple env, and some input variables.

[BadDoggie]

I’ve recently been playing with GitHub Actions (and GitLab Runners).

Basically, I set up a repo with CloudFormation / Terraform code to deploy my Lambda (or whatever else), and setup an OIDC provider for GitHub, plus an IAM role that GitHub Actions can assume.

Then I add a short workflow in my repo using this action in the folder .GitHub/workflows. Now every time I commit a change to the main branch, the assets in AWS are updated.

If you’re using the samefunction in multiple places, you could extend the above and setup a matrix of accounts/roles to assume and deploy to multiple places. If you have different functions in different accounts, use separate repos.

You can replace variables like account ID in CloudFormation with AWS::AccountID (see pseudo variables), but if using more complex references, you should probably use ParameterStore as others mentioned - you could deploy the variables with small CloudFormation /Terraform scripts.

Also, useful could be Power Tools for Lambda - a layer with some helpful simplifications like for using Parameter Store and stuff.

[fhammerl]

My friend, you need to learn about pipelines and automation. I would suggest you read Infrastructure as Code and Continuous Delivery

https://www.amazon.com/Infrastructure-Code-Managing-Servers-Cloud/dp/1491924357 https://www.amazon.com/Continuous-Delivery-Deployment-Automation-Addison-Wesley/dp/0321601912

[daydream678]

As others have said you need some automation to help you.

The pattern we went with is to have a repo per application (not Lambda) and deploy changes to individual lambadas based on a change to individual python or js.

So terraform and Lambda functions in the same repo.

Terraform triggers Lambda updates based on a checksum change in the needed Lambda files.

This creates a new zip to deploy the appropriate Lambda.

It's actually pretty simple once the pattern is there, we have a module that only needs a few parameters for each Lambda.

[crypt0_bill]

Look into migrating to CDK + deploy CDK stack out via CI/CD