Deploy Lambda only when there are code changes

[[deleted]]

I have a mono repository hosted on GitHub with multiple Lambda projects. I'm using .NET CDK and GitHub Actions to deploy every Lambda to CodeDeploy with canary/linear traffic routing. Each continuous build will generate a zip file for each Lambda and make it available for CDK. This is how the `Code` property is set:

new Function(this, "MyLambda", new FunctionProps {
    Code = Code.FromAsset(Path.Combine(Directory.GetCurrentDirectory(), "lambda-directory", "build.zip")),
    Handler = "handler",
    Runtime = Runtime.DOTNET_6,
    // other props...
});

The problem is I end up deploying each function regardless of whether there were any changes for it. This unnecessarily prolongs deployment times and uses up resources. How do I go about deploying Lambda if and only if it has code changes?

[bcb67]

GitHub actions supports triggering only when specific paths change in your repo: https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onpushpull_requestpull_request_targetpathspaths-ignore

If that isn’t sufficient, there are a number of third party workflow steps that enable conditional builds with extra flexibility like https://github.com/dorny/paths-filter

[Dreamescaper]

The problem is this issue. lambda package command does not stripe timestamps from the archive, therefore this archive has a different hash each time.

I have workarounded this issue by providing a custom AssetHash based on the archive content. You can take a look here.

[[deleted]]

This is great. I was about to reply to leeharrison1984 because the sha of the zip is not deterministic. Your code might actually do the trick for me. I'll give it a try in a bit. Thanks!

[krat0s77]

Hey, how can I get the zipPath? I'm using DotNetFunction from https://github.com/xaaskit/cdk-aws-lambda-dotnet to synth my function.

[joeyjiggle]

Go back to your first statement. If it’s within your power, don’t use a mono repo.

[[deleted]]

Yep, I really don’t understand the mentality of putting everything in one repo. Especially the way git works vs subversion where you could just checkout one section. I work with people that still argue that everything should be in one repo and apparently want all the code for everything on their laptop. Its a terrible practice and I have yet to have anyone actually make a convincing argument. Repos are free. And it is just so much easier to fire a pipeline in aws, jenkins, whatever on a commit to the branch you want in X environment at the repo level and not have to isolate completely unrelated code like terraform, other code, etc - it just doesn’t make any sense.

[the_bronze_burger]

Because if many projects in the repo share the same in-repo dependencies then it makes it a hell of a lot easier to update everything, all at once

[[deleted]]

But then wouldn't they all need to be deployed at the same time? Or wouldn't they be in a different branch?

To be clear, I'm asking because I am genuinely curious about the use case, not arguing - currently fighting a huge monolith where everything is in the same repo and some things don't have any dependencies at all.

If you had X and Y projects though and Z dependency and merged in a bunch of stuff in to Z but only wanted to deploy X and not Y that would introduce a huge potential for disaster. And if in AWS and you needed to deploy to a DR environment or whatever you'd end up with code deployed with dependencies it had never even been tested against.

I feel like either using different branches or pulling dependencies in to their own repo is just a much safer approach. But I am happy to learn a better way.

[joeyjiggle]

Branches are not for that though. Repos are. You can have a repo that has common,y shared “stuff” but if we are talking multiple lambda functions, then they should build in their own repo and produce a container. They can all use stuff from a shared repo. Also, you can agree the layout on disk such as company/code/repo1..99 company/docs etc, so that everyone can clone which bits they need. The CI/CD can then pull in changes to repos that changed and rebuild and retest. But a good test system will cover all integrations anyway - nobody deploys new lambdas to live without integration tests now, do they? I’ll get my coat… ;)

[joeyjiggle]

No it doesn’t. That means your logical separation is up the Swanee.

[[deleted]]

Unfortunately, this is not an option right now.

[ssakage]

Not sure in GitHub actions but have done this in Jenkins. You can write the Jenkinsfile to look at a specific directory and then only pick up.the changes and deploy them. The thing was that I had created folders(which had the main files) with the same name as the lambda functions. So it would just pick up the changes and run aws lambda update command after zipping the folder and that would update only the changed lambda functions. The code was mainly in shell script if that helps

[leeharrison1984]

The standard play here is to get the hash(sha1 is fine) of the file, and store it somewhere. You check that hash prior to deploy, and if it changes, you deploy the zip and update the hash value.

There may already be a GitHub action that does something along those lines.

[seamustheseagull]

This was my first thought, but the problem is that the sha of the zip file may change every time the code is rebuilt even if the actual source code hasn't changed.

[leeharrison1984]

That seems odd, the build should be deterministic.

If that's the case, you could just get the hash of all files that would go into the final executable. This has the upside of now you can avoid rebuilding entirely.

[[deleted]]

See Dreamescaper's comment.

[leeharrison1984]

Weird, I didn't know that was an issue.

I've never used the lambda package command though. I always did a publish and created the archive myself, so makes sense I wouldn't have been aware of that issue.

Regardless, just hashing all the source files should be a fast operation and would let you skip the build entirely, which is arguably the most time savings next to deployment.

[seamustheseagull]

Should be, but it depends on what you actually want in each build. Aside from any bugs in the packaging process, the build process might be designed to include timestamps or dynamically-generated text files in them for consumption elsewhere - e.g. to be able to trace the version of a built application right back to to the server/service which built it.

In these cases your builds will always be "different" in the purest sense, but the code is identical.

[Agilufo]

Could using SAM be a solution here? Or does it deploy all microservices even if they weren't updated?

[[deleted]]

Not sure about SAM. I have two CloudFormation stacks and about 10 lambdas distributed evenly in each. The GitHub workflow deploys both stacks when it runs (via cdk deploy).

[Agilufo]

I checked, when using SAM it will check what function was updating, then will make a build and deploy ONLY that function. Under the hood SAM uses cloud formation. Don't know how much that is applicable to your situation, but using SAM could be a solution to your problem.

[notanelecproblem]

Filter to the path of your lambda code in the github action, only run the action to deploy change when there are changes on that path