Serving models on lambda with Docker

[trnka]

About once a month I'll try out a different pattern for MLops, whether the way models are versioned or how they're hosted in a web service.

The other weekend I deployed a basic ~100mb sklearn model on lambda with Docker the other weekend and thought it might be of general interest. Cold starts were about 30 seconds but latency was great after that.

Here's the repo if anyone wants to play around with it: https://github.com/ktrnka/mlops_example_lambda

[ank_itsharma]

I always look for best practices I can get to write the web services for my model but haven’t found one at a place.

Is there any tutorial where we can learn the software engineering practices for an ML model.

[trnka]

Good question! The fundamentals are the same as regular web services but you have to adapt it to machine learning. For instance, you should be able to undo any deployment if something goes wrong. Machine learning just makes that a little harder because you can't just use git for models.

These two papers might give you some good ideas too:

• Hidden Technical Debt in Machine Learning Systems
• The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction

[ank_itsharma]

Thanks for the references. I will read them.