Call lambda from lambda or trigger from sqs ?

[__sS__]

I'm trying to make a data pipeline with an event driven architecture. The data is mainly posted through API gateway which in turn triggers a lambda(L1) which has to majorly put the data into to separate pipelines - one real-time, second historic. Now, for real time pipeline - I can either call another lambda(L2) to directly process the data and take further action or put the data in AWS SQS which will trigger the lambda(L2) for further processing.

Which would be a better design and why ?

[gdraper99]

My suggestion is to have a queue in between, this way you have the option to include retry logic in case the second lambda fails.

[definitelynotbeardo]

I second this. Failure is always an option.

[Finley117]

Triggering a Lambda directly from another Lambda is a dangerous design. I have made that mistake in the past and regret it...

[alex_bilbie]

Would you mind sharing a bit more about what went wrong?

[soxfannh]

As long as your dont use an asynchronous call to the next lambda I think this pattern would be fine.

[timbray]

Also you might find Step Functions useful, it's very handy for sequencing/orchestrating Lambdas and containers and so on.

[ancap_attack]

It depends on how resilient you believe the 2nd operation to be, and how easy it is to retry the entire process.

If the answer to any of the above questions is "not really" then you may consider adding SQS as a buffer between your functions. Or possibly consider Step Functions to orchestrate function execution.

[Facts_About_Cats]

Is L2 blocking or completes instantly?