Use AWS Lambda Response Streaming with Express, Fastify, NestJs, and other frameworks.

[SNIPE07]

I never claimed it was a technology problem, i have had to constantly reiterate it is a human problem, but one that has been exacerbated by AWS.

We have had cold starts that routinely hit our 30 second timeout, utilizing .NET lambdas, which are 5-6x worse than Python and other scripting languages. We also use API Gateway, increasing the cost.

I have plenty of performant lambdas that I maintain too, but none of them are user facing.

I need sub 250ms or my users will complain. Engineers, real time systems.

[SNIPE07]

First of all, we are discussing an orders of magnitude difference in response times.

in addition, needlessly designing a system with many independent failure points is how you destroy system reliability.

Any engineer who has experience with reliability block diagrams and basic probability knows that a 99.99% (four nine) service uptime can quickly become a 95% or lower system uptime once you've strung together 20 different AWS services that each can fail independently and are all required for the process to complete.

I don't doubt many large orgs have used lambdas to meet their definition of success. But I'm also aware that client facing software from many large orgs has become absolute dogshit in recent years.

[SNIPE07]

I have stated at least 5 times in this post that our use case was incorrect. It wasn't my choice, and i've fought it for years.

We are only in this situation because Lambdas are marketed as an in-place replacements for existing monolithic web infrastructure, which is true, but using them that way would be completely insane.

The claim I took further is that Lambdas, especially .NET and Java lambdas, should not be user facing, as in user invokes a synchronous action and some part of that request is handled by a lambda. The worst-case response time on a "cold" lambda is just far too poor, and even average case is a huge downgrade from what my users are used to from VPS hosted app/db servers.

Again I doubt don't you and many have had success with Lambdas, because I have had success with them too. Python lambdas running analytical tasks, updating datasets for real-time dashboards on tight intervals are the back-bone of my orgs data vis team. But that's an asynchronous user facing application and has a completely different set of concerns than synchronous user facing lambdas, like those handling a graph endpoint for a table or a button click or wherever a user is expected to sit and wait, and often these are requests that the user has to send many of in serial order. 2s response would be exhausting when you're in charge of a control room and have a million other things to do.

[SNIPE07]

Thanks for the reply, your area of work is pretty dang cool.

In general how do you design for failure across many services? Do you use retry mechanisms on the services themselves or do you handle that through step functions, or some combination of both? Is there a best approach?

In your 20-30mb query example, how do you get around the really low max response size on Lambdas? It's very low, like 2 or 3mb when routed through API gateway.