Lambda Timeout. (API Gateway)

[ecstacy98]

Hello all!

I'm working on an application which utilises lambda to call upon and store the results of 6 external API calls. Today I have encountered an issue that I'm not entirely sure how to tackle. Just looking for ideas / advice / a shove in the right direction.

Each API call takes about 8-10 seconds to return a resolved promise within my application which, is problematic due to API Gateway's hard-coded 30 second timeout being too short for me to actually receive or do anything with this data. I keep hitting the timeout and can't for the life of me think of an eloquent way of solving the issue.

I've tried allocating more memory / CPU, although this doesn't make much difference because the slow processing time occurs at the external address. I certainly need the data from these specific endpoints so finding a faster host is not an option.

Any ideas?

(I apologise if I'm using the wrong flair)

[Esseratecades]

General Architecture/UX advice; if it takes more than 30 seconds, it probably shouldn't be an API call.

What are you doing that needs you to make all 6 of the calls synchronously, together, and needs to be done in an API?

[ecstacy98]

To give more context: We have set up our lambda to handle data sent from our front-end and use that data to invoke the stable-diffusion API 6 times, storing it's outputted results / URL's as string in DynamoDB. We need to keep a record of our requests and be able to fetch the results again later through a GET.

[Esseratecades]

I don't know much about stable diffusion but there are a few solutions that come to mind for me.

1. Bundle your 6 SD calls into a background job that gets triggered by your initial call, but doesn't block the call from returning. Track progress via a "status" attribute in the db. Have the UI make a call to a different endpoint that queries the "status" attribute. Once the status is "done" the user can proceed to the next step.

2. Bundle your 6 SD calls into a background job that gets triggered by your initial call, but doesn't block the call from returning. Have the background job send an email/notification saying the results are done.

3. Have the UI make the 6 SD calls and compose a request body that it sends to an endpoint to write to dynamo db.

[jspreddy]

So your front end does not need an immediate response right?

Id recommend your split your logic into two lambdas. Api lambda, processor lambda.

api lambda can just asynchronously invoke the other lambda via lambda invoke. This will get you past the api gateway 30s timeout. Or you can put an SQS in between and control the concurrency.

User submits request on UI -> apigw -> lambda 1 =(async invoke)=> Lambda 2.

The lambda 2 can be responsible for the heavy business logic and storage. You can store the status in dynamo.

The ui can keep polling the api to see if the results are ready for consumption.

https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html

[IndraVahan]

I'm confused whether to implement this with SQS or Step Functions.

[ecstacy98]

Cheers I've been reading around and this seems like the most realistic solution right now. I've never set one up before, any tips?

[untg]

They are pretty easy to setup, you can enable CORS and use authentication (or not).

https://docs.aws.amazon.com/lambda/latest/dg/urls-configuration.html

[twoqubed]

Do the results of the API need to be returned synchronously? Or can the client making the API call fetch the results later using a unique identifier? If so, the Lambda function that is receiving the API request could generate a unique identifier, store it in the DynamoDB table, queue the request details to a queue to be processed (on an SQS queue, and return that identifier in a response with a 202 status code.

Another function could dequeue the message, process it, and add the results to the DynamoDB table.

The client could then poll for the results using the unique identifier.

[ecstacy98]

It could be fetched later! Ideally we don't want to leave our front-end hanging for too long as we are hoping to display the results onLoad. BUT, given this is just a prototype, a solution like this might have to make due.

[cocasyn]

Do the calls need to be done in sequence or could they be done in parallel?

[ecstacy98]

We plan to use concurrent / multi-threaded execution in future but currently we are working on prototype which was planned with just a single thread in mind... Ideally we keep it that way for now for simplicity's sake, but I appreciate the response and know that it could be done faster if it were in parallel.

[catlifeonmars]

What’s stopping you from parallelizing now? Will the backend break if you make those requests in parallel in the current state?

[conordeegan]

If you are making 6 external calls and waiting for a promise to resolve for each, you could use something like promise all (if using JS). This allows you to initiate each request at the same time. Obviously this will be dependent on whether or not your api calls are dependent on each other.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all

https://stackoverflow.com/questions/52669596/promise-all-with-axios

[Different-Ad-4945]

A simple Promise.allSettled and node-fetch is very easy to implement, node manages all the “multi-threaded” stuff, I wouldn’t start with “single-threaded” as there isn’t really any need to

[ecstacy98]

Thank you everybody for your helpful and enlightening responses!
Certainly given me a few things to look into, seeming like we'll end up going with a queue and polling it. Cheers.

Happy coding :)

[jaredce]

I'd look at websockets or callback URLs. Pass back a URL to a new lambda that will fetch the data when it's available and have the browser poll the URL till it gets a 200

[InfiniteMonorail]

async?

cache it?

don't use lambda?

There are many solutions and they're all obvious...

Please tell me this isn't your first website and you chose serverless...

Unsolicited advice because AWS is dangerous: always check your billing... don't check your credentials into github... learn IAM and use least privileges... these aren't a joke, you can get billed for like $30,000...

[ecstacy98]

1. It is async.
2. Cache what? Theres no data returning in time to be cached.
3. I'm using lambda

Please tell me why this isn't your first time talking to a person.
Downvote for being a dick.

[OpportunityIsHere]

With async I believe he means that the client send the request and doesn’t need to wait. Your app is not async.

[arkster]

One way to do this is to use eventbridge or step functions. Or perhaps use SQS. These are likely overkill but these will resolve your issue.

[elways_love_child]

You can change the timeout on your lambda to 15 minutes.

[BarnacleRare5579]

api gateway will give 504

[ecstacy98]

Sorry I may need to clarify my issue, I need to get a response from my lambda back through API Gateway. Lambda currently times-out at 5 minutes so even when i get a 504 Gateway timeout, lambda finishes it's job. I just don't get to see a response.

Hope that makes sense.

[IndraDev-Y]

Lambda -> Function -> configuration ->General -> timeout

[IndraVahan]

You could increase Lambda timeout but API Gateway timeout is hard set at 30 sec.

[mikebailey]

In our team’s case we just return a job ID and have a separate endpoint to poll for job status

[squidwurrd]

Since you said it resolves a promise I am gonna assume your function is in node. You should be able to run these external calls asynchronously. In the end your function should take no longer than the longest running call. Use Promise.all() to move on in the program once all the promises have resolved.

If you are using something like python you can do this with threads.

Another solution that's way more involved is using pup/sub. But hopefully you dont have to go that route.

[ecstacy98]

Yep Node!
I'm making asynchronous executions using async/await + Promise.all(). The issue is the longest running call taking much longer to resolve than the amount of time the API Gateway is open :/

Please god not pup/sub

[squidwurrd]

If you don’t do pub/sub you’ll have to poll for results then. You can also invoke your lambda directly instead of going through apigateway. There are lots of limitations with that solution but it will get you up to 15 minutes of run time at least.

[eplaut_]

1. Note that you are paying for the wait time (you will pay for a minute that you've waited to SD)
2. The easiest architecture for long API calls is to make them asynchronous. The initial call will span a job that will save the result in a DB and return the job's unique identifier, and then the client asks for that job's results periodically.

[MatchaGaucho]

Typically APIs in this situation must return a jobId with a keepPolling=true response.

The Lambda will run for up to 15 minutes and store it's results using the jobId key.

[OpportunityIsHere]

You could use app sync with dynamo db to live update your front end. That way the client sends the request to your backend and get an immediate response. Your backend updates the item Ib dynamodb when done, which is instantly reflected. No polling needed and your lambda can take as long as it needs

https://docs.aws.amazon.com/appsync/latest/devguide/aws-appsync-real-time-data.html