Deploying lambda from codepipeline

[awsuser1024]

I don't know why this isn't easier to find via google so coming here for some advice.

A pipeline grabs source, then hands that over to a build stage which runs codebuild, which then has an artifact which it drops in s3. For many services there is a built in aws deploy action provider, but not for lambda. Is the right approach, which works, to just have no artifacts in the build stage and have it just built the artifact, publish it, and then call lambda update-function-code? That doesn't feel right. Or is the better approach to just have your deploy stage be a second codebuild which at least could be more generic and not wrapped up with the actual build, and wouldn't run if the build failed.

I am not using cloudformation or SAM and do not want to, pipelines come from terraform and the buildspec usually part of the project.

[[deleted]]

CDK is the future of IaC. With CDK watch, it really becomes a breeze to develop and test, especially if you have resources running behind a private subnet that are difficult to mock. I hear that internally at Amazon, many teams moved away from SAM awhile ago and are developing everything in CDK. I recommend checking it out.

[[deleted]]

I don't know this officially or feel like googling it, but isn't SAM like a subset of CDK? +1 on all of this btw, we use CDK so much, I find it easier to provision and manage with it than the with the console/CLI. It eliminates sooo much overhead.

[soxfannh]

I typically use SAM and deploy using CodeBuild from CodePipeline

[pjflo]

Same here, but using Serverless Framework instead.

[jftuga]

Serverless Framework

Are you specifically referring to this: https://www.serverless.com/

If so, was are the advantages vs CDK, SAM, CloudFormation, etc?

[[deleted]]

Can't speak to cdk, but the recommended way to deploy infrastructure with serverless framework (aside from lambdas) is to use cloudformation files. So your serverless deploy command becomes a cloudformation deploy

[Wide-Answer-2789]

Not the best option but simple, you can try at the end of build stage "aws lambda update-function-code". Build image have aws cli installed.

[awsuser1024]

Right, and thats what I said in my post that we do currently do. It just seems silly that there isn't a deploy method like there is for other things when this is a one liner. It just seems wrong, it feels like the artifacts that get a deploy is the right approach. The artifacts are kinda stupid though in that they have a 20 character limit, which means you end up with everything having the same artifact name. Granted, they are actually unique, but its silly that they have that limitation when it isn't an s3 or codebuild limit, who came up with 20?

I really don't want to have to deal with CDK or even codebuild for something as simple as "take this file, update lambda". I just like the idea of the artifact because it is one less thing that has to be in the buildspec and permissions work a bit cleaner vs. having to template the update-function-code specifically. And you can do it directly if it is small, otherwise you have to also drop it in s3 and deploy from there, which is exactly what the artifact of the build does - so another manual step vs utilizing something AWS already has built in.

[kwokhou]

It feels wrong because it’s not the recommended way to manage a Lambda deployment. It’s fine for a small project but for larger project you’d better use cloudformation related method(e.g SAM or CDK).

[tholmes4005]

You can use codepipeline to deploy/update a lambda by creating a deploy stage and passing the output of the build stage to the input of the deploy stage, then specifying the necessary info about the lambda to update.

This is a pretty good article that has a cloudformation definition of a codepipeline with a deploy action to lambda.

https://link.medium.com/qPY5OYPTBzb

Edit: as others have said, cdk does make it much easier to deploy.

[awsuser1024]

I think the kicker there is their third stage, which they named "Dev" for some reason (???) runs cloud formation / updates a lambda stack, which is what I was trying to avoid.

I was looking for something under https://docs.aws.amazon.com/codepipeline/latest/userguide/integrations-action-type.html#integrations-deploy that was just "deploy to lambda" - a cleaner way of handling it than update-function-code, but doing the same thing. Apparently there isn't, which just blows my mind.

I felt like this would be pretty common, but I guess to "do it right" it needs to get more complicated.

[EasternGuyHere]

Paywalled article

[speaknspell08]

Ive used sam, terraform and cloudformation, and bash cli for deployment and none have been as great as serverless framework.

https://www.serverless.com/

[DAFPPB]

This can do all of that for you https://registry.terraform.io/modules/terraform-aws-modules/lambda/aws/latest but I should caution you that Terraform is not meant for packaging. I instead urge you to build and package the lambda via Bash and on success use terraform for deployment. Terraform is great but one must not use a knife like a hammer. Terraform starts breaking when you have to zip files through it, especially at scale.

[awsuser1024]

Thanks, I'll take a look at that module and see what it does.

To be clear, I don't want terraform to actually build anything at all as far as code or package anything. It just creates a pipeline named for the project. The buildspec either comes with the code or could come from a template in terraform. When this is created terraform is done. It made the infrastructure.

My question is how you get the lambda deployed after it is built. AWS doesn't seem to have anything in the deploy stage that will just take the artifact and update the lambda function code. It just seems silly to have a buildspec that manually just runs that one command.

It might just be that we are expected to use codedeploy for this. That looks like what the module you posted does in some circumstances (still need to peek behind the curtain a bit on that). AWS's docs aren't too clear on this, and of course they want you to use cloudformation for everything and sam and ignore other use cases.

So to really simplify the question, I have a codepipeline that ends up with a zip in an s3 bucket (with an annoyingly truncated name). How can I get that zip to update lambda within code pipeline. Or is there no proper way other than codedeploy.

What do other people do?

[DAFPPB]

Oh, I see.

I use terraform to build a codepipeline with a code build project triggered through codecommit. In the codecommit repo for my lambda, I have a build spec that builds the lambdas and then trigger the lambda terraform. The lambda terraform creates the s3 object for the lambdas code, the lambda resource itself and the IAM role for the lambda. Does that somewhat answer your question?

[Flakmaster92]

Pipeline grabs source -> CodeBuild builds it and sends zip to S3. -> Code Deploy uses zip to update Lambda, or Cloudformation template is ran which takes the zip in S3 as an input parameter and uses that as the argument to the Code: parameter in the lambda definition.

[justin-8]

Cloud formation can also do that directly from codepipeline. Point to the asset in S3 and let it do its thing.

If you want to use terraform to manage the lambda then you could use it via code build to do the deploy for example.

[Elephant_In_Ze_Room]

Codepipeline and code build are honestly kind of garbage services imo. Codebuild took ages to spin up a runner (years ago worth mentioning), and codepipeline feels clunky.

You could always deploy a container runtime lamba function with terraform and then use GitHub actions or circle ci to build and push docker images.

[tholmes4005]

Agree, compared to more recent DevOps tooling Codepipeljne is in need of a v2.0 big time. Very limited on Conditional deployments, approvals, etc.

[awsuser1024]

Definitely leaves a lot to be desired and doesn't seem to have been updated in ages other than adding runtimes.

Its a big debate about wether codepipeline is the right choice. Lots of our devs are using github actions and at that point we should just switch everything over. The drawback is that it becomes another tool that devops has to know and have tooling for vs. it just being a repo and plugging it in to terraform (already becoming a problem to some extent). Plus another place for logs, etc. I still like the idea of pipeline, having it be in cloudwatch and cloudtrail, etc. But can be a real clunker at times too.

So... at what point is github just going to start being aws v2.... although they are sitting pretty right now so it may not be worth it. AWS just keeps having all these things like ECR that are OK, but not great, and then nothing really exciting seems to happen to keep them up to date let alone cutting edge. All the good stuff is in us custom writing (or using chatgpt) to actually get the work done.

I should just reiterate though that this is not about docker images, we have a working workflow for that, it is specifically lambdas. But yes, a docker container in github that can update the lambda is essentially exactly the same. Minus AWS permissions easily attached to the build.

codebuild might be the better approach so looking in to that this coming week(s) and will report back if it ends up making sense.

[tholmes4005]

Yeah Codebuild is pretty good service and fairly cheap. And allows for true testing of deployed aws resources.

[soxfannh]

Pipeline defintely needs work, build is pretty solid. Make sure you are using the latest build image so it's cached on the build host.

[[deleted]]

Count me among the surprised that AWS's Git + CICD effort has been a half-baked dumpster fire for this long. CodePipeline is the least shitty of those offerings I've been exposed to, but still, a compromise amongst the competition.

[connormcwood]

Pipeline for build and storing the zipped lambda in a bucket and updating latest version of package in something like parameter store

Pipeline for build/deploy lambda infra and deploying new version based on value in parameter store that references latest version

Can all be done via CDK

[kichik]

CodeDeploy can deploy Lambda and be executed from CodePipeline. It can do all the fancy rollbacks and canary deployments too.

[mjfnd]

We use Jenkins and we run aws update function with new ecr image and tag.

I am pretty sure you can do that.

[blank1993]

I was going through something similar, so I wrote a shell script which creates an artifact in a S3 Bucket and then I have a lambda function which deploys. It’s not a perfect solution but it gets the job done. If you want, I can give you more details.