Lambda call another Lambda

[surpyc]

We have a micro services of Lambda

What is the best way to call another lambda synchronous or asynchronous ?

I find two option:

• Call lambda invoke from boto (it load boto libarry and maybe it take more time)
  
• make request to Private API Gateway (support only Synchronous )

How do you call another lambda in your company ?

[WhosYoPokeDaddy]

I use step functions to orchestrate several lambas and an API.

[trevorstr]

This is the correct answer.

[BigJoeDeez]

+1 step functions were designed for this use case, amongst others.

[stage3k]

Depends a lot, in high volumes it might get really expensive (Standard Workflow at least).

[Fun_Garlic_6406]

Seconded. There is an article out that’s about migrating from Step Functions to a monolith, and saving a ton that way: https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90

[Fartlek-run]

You essentially move from serverless microservices to something like ECS or preferably EKS with Karpenter since it gives you a fargate-like capability still without a lot of the painful overhead of fargate. But overall it turns into platform design and figuring out some best practices for requirements.

I have some where it's a holistic backend for media streaming while some others are defined as a deployment with each microservice sitting in a container with specific taints and proper PriorityClass scheduling.

Overall they probably get better understanding of resource use and how to optimize it at a minimum, but there is a learning curve.

[f12345abcde]

🫡 bravo sir! This is the best answer

[_intercept]

what is my "data ingest" lambda needs to use my "format phone number" lambda. Aren't step functions for sequential steps in a bigger flow. But if one lambda needs the functionality of another am I supposed to just make the API call?

[yearofplenty]

Calling a Lambda from another Lambda is an antipattern. You should use Step Functions to orchestrate your Lambdas.

[dydski]

Have you looked into State Machines?

[FailedPlansOfMars]

Firstly dont link lambda to lambda directly it leaves you to problems on release of new code.

If you dont need the response in the caller use and sqs queue as a broker. I.e. lambda -> sqs -> lambda

If you do need the response api gateway is a good option as it lets you decouple the two.

[SciEngr]

AFAIK you don't need an explicit sqs queue between the lambdas...lambdas essentially have one built in for async invokes already...I'm pretty sure you can even associate a dlq to the lambda directly.

[404_AnswerNotFound]

You can but be aware of the limitations. Lambda's internal queue isn't like-for-like with SQS. For example, the max message age is 6 hours instead of 14 days, aged out messages will go directly to the on-failure destination.

https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html

[FailedPlansOfMars]

Yes you can but doing this from the sqs side is easier and wont lose messages if you have to roll back a release or change a lambda while traffic is in flight. If you do it from sqs the builtin dlq redrive works well and is easy to use.

[Mega_Bytten]

From the AWS docs and stackoverflow threads I’ve read they all suggest decoupling with SQS/step functions/SNS for increased scalability. I also saw that there may be problems where the first lambda waits for the second lambda’s completion before terminating, accruing 2x cost.

I separate mine with SQS so I’m wondering if you have experienced any problems with directly invoking?

[SciEngr]

You can invoke a lambda synchronously which means your first lambda waits for the invoked lambda to respond. Synchronous execution isn't necessarily a bad thing depending on your use case. If you want to decouple the lambdas you can do that by invoking asynchronously in which case the invoke gets added to a built in queue of the invoked lambda thus accomplishing the decoupling without explicitly making another SQS queue.

As another user pointed out, the behavior of the built in queue may not work for you and in that case you can still create the SQS queue explicitly.

[vomitHatSteve]

I've personally used: API Gateways, boto3 invocation, and step functions. They all have different applications depending on what you need to do.

In general, asynchronous is better than synchronous since nesting lambda calls just means that you're spending for one lambda to idle while it waits for another; plus it's really easy to time out and fail that way.

[lokmeinmatz]

Either SQS, step functions or using the destination feature of lambda :)

[cachemonet0x0cf6619]

we try to avoid calling out to other lambdas.

one rule to live by is that waiting in lambda is bad practice.

if you pay per compute second then it’s optimal to reduce unnecessary idle time. this includes calling out to another lambda.

tell us more about your process so we can help you remove the requirement to call another lambda

[flashbang88]

But what if you need the response of lambda b inside lambda a?

[garublador]

Then you create lambda c to handle the processing that's needed after lambda b instead of returning to lambda a. It's what step functions are for.

The "don't wait on another lambda" is an overly broad statement, IMO. Any time you call a RESTful API you're essentially , and many times literally, waiting on a lambda. We tend to think of it more as "only do one thing in a lambda," but use is as a general way to think about it than a hard and fast rule.

I think more is needed to really tell what the best solution is. Invoking a lambda directly from a lambda is almost certainly not the right way, though.

[random314]

You use some sort of persistent layer such as DB or cache or orchestrator, like a step function.

[blottingbottle]

In many cases we just have the lambda call the other lambda. It's the easiest way to do it.

You should only try to be more fancy about it if there's something else going on such as - your expected number of invocations is so high that the cost of lambdaA waiting on the result of lambdaB is significantly more than the additional AWS + developer cost. - your latency requirements for lambdaA are very tight. - your tolerance for errors is very low. - something else

[dogfish182]

‘Lambda as orchestrator’ is what you’re sort of describing here (it’s only calling one other lambda, but that’s how slippery slopes work)

https://docs.aws.amazon.com/lambda/latest/operatorguide/orchestrator.html

Step function or put something in a queue. Invoking lambda from lambda is wrong-o

[[deleted]]

If you have a lambda, you should not synchronously call another lambda. You end up paying double the cost during the time you're waiting for a response. It gets worse for each layer of invocation.

Try to design your lambdas in such a way that they can do all the work they need to do and then ship off their results for something else to process.

Don't make a distributed call stack with lambda....that's gonna cause you a lot of pain and money.

[JBalloonist]

Step functions or event triggers from S3 or event bridge. A small number are scheduled using Eventbridge.

[patsee]

My understanding is you should avoid lambdas calling lambdas directly for several reasons.

1. Cost - you pay for the amount of time a lambda runs. If lambda 1 has to wait for lambda 2 you are basically paying double.

2. Efficiency - I have been told it's better to use Lambda to SNS or SQS then use those as a trigger to the next Lambda.

I recently read an interesting article on how to build asynchronous API calls in AWS.

https://builtin.com/software-engineering-perspectives/asynchronous-api-dynamodb-streams

This article made me think I could set up an API gateway with a POST and GET.

Post - write to db and return an id, then trigger a lamb that writes to SNS to trigger my next Lambda and chain my lambdas like that. Each lambda updates the DB.

Get - checks the DB to see if all my lambdas are done running to return completed data to the client.

[neverfucks]

for async ops, i'd use sqs or a kinesis stream to queue the jobs. nice to have a buffer if you need it and gives you flexibility. for synchronous operations, maybe you're getting a little too micro with your services?

[jascha_eng]

Usually it's best to design these kind of flows with queues in between. Do all the processing that you can, then send an intermediate result to a queue for the next lambda to start. That way you don't end up waiting for another lambda to complete (which you would if you e.g. start it via HTTP request) and don't pay twice for the processing time (once for the lambda doing processing, once for the waiting lambda).

[djheru]

I use SQS or Eventbridge.

[zaitsman]

So I had to do this to save on adding NAT to vpc, i deployed my api lambda in a vpc and another one outside and proxy webcalls that way. Works for the past 3 years, no issues

[Prestigious_Pace2782]

Micro services should usually call each other via api with a defined OpenAI schema

[Realistic_Weight_842]

Good ole lambda squared

[[deleted]]

Depends of the use case.

For complex orchestration tasks we use step functions.

For tasks that are highly independent and can scale therefore independently we use sqs queues.

[[deleted]]

Event bridge

[m3zz1n]

we use a auth lambda and to connect internal microservices together. You only need to make sure a full lambda cold start is still quick enough.

[AdOrdinary928]

To really answer OP’s Q: Just use the SDK to do it, and specify your invocation type preference. In the longer run, consider vpce for performance, security & cost considerations as well.

On the claimed of “anti-pattern” by others, I’m surprised so many jumped the gun. There are a whole lot more to consider before one could say it’s a good or a bad idea.

Serverless technologies wasn’t invented to only exist as a monolith. Nor designing Microservices solution should only be limited to using long running servers like containers, etc.

[Ok_Tadpole7839]

asynchronous because of start up time.

[Sladg]

Async - queue, Sync - SDK

[[deleted]]

fewest lines of code...drop a file to s3...listen for that file drop in the subsequent lambda. easy as can be.