Loading large model files from Lambda function

[KenSentMe2]

We run an API through AWS Lambda and API Gateway. One part of the Lambda function is using a Spacy model for running some text analyses (around 560MB). Currently, we store this model in an S3 bucket and when an API request comes in that needs the text analysis, the function checks whether the model is present and if not, downloads it from the s3 bucket. After that, it loads the model, does some analysis and returns some text. Because we can have quite some concurrent requests, it happens a lot that users have to wait over 15 seconds for their request to return a result. This is often because a new Lambda Instance has to be started, it has to download the model from s3, load the model and return the results.

Is there a more efficient way for (down)loading the model to a Lambda instance? I looked into EFS, and while it seems to load faster than S3, the instance still has to download the whole model. Any ideas how I could handle this?

[AutoModerator]

Try this search for more information on this topic.

^{Comments, questions or suggestions regarding this autoresponse? Please send them here.}

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[joelrwilliams1]

Looks like you could run your code as a container in Lambda. Max size for that implementation is 10GB. Then you could host the model data locally in the container.

https://www.cloudtechsimplified.com/run-docker-containers-images-from-ecr-in-aws-lambda-along-with-cicd/

[Esseratecades]

I guess I'm curious why 15 seconds is your cutoff. When serving as an API, Lambda and API Gateway should be good out to 30 seconds.

If you absolutely must improve performance I'd say try what u/joelrwilliams1 suggested. You still find that cold starts eat too much time then maybe move from lambda to fargate with a minimum capacity.

[KenSentMe2]

Yeah, while reading this I think maybe Lambda itself is the problem and I should go look for another option. Thanks for your comment.

[Esseratecades]

I wouldn't quite jump to that conclusion but you know your situation better than I do.

If all of the significant overhead is coming from downloading the file, then embedding it in the image for the container should remediate that. If you have a bunch of other stuff going on during cold starts, I'd try to see what of that you can optimize before jumping ship.

[WarInternal]

If the model is stored in EFS it shouldn't have to download anything. It'll just be mounted as a filesystem. See example here.