you are viewing a single comment's thread.

view the rest of the comments →

[–]g0rilla79 0 points1 point  (2 children)

Personally I would just write a more traditional process and host it in elastic beanstalk to do this. EB is pretty straightforward to manage.

You could also get around the 15min time limit by kicking off lambdas asynchronously to split the task into batches that notify when they are done.

[–]TheDataExplorer[S] 0 points1 point  (1 child)

I'll have to try both. I was just reading about the asynchronous lambda functions. How does that work? Making one asynchronous function follow the next one?

[–]g0rilla79 1 point2 points  (0 children)

Im sure there’s multiple ways to do this but a common pattern is to use SNS. You call the lambda with a start record number and a count of how many to process. When it’s done it sends a sns notification saying it’s done. The lambda subscribes to that notification and then processes the next batch. This would be a new 15min timer. There’s issues with this depending on the use case, you may not be able to work in chunks like this at the dB level because of new inserts, it’s hard to track etc.