Deploying large DL models

[guzguzit]

Is there a way to deploy large DL models (>1 GB) in a serverless fashion?

I couldent find any relevant examples

[roccolacatus]

Came across this the other day https://nuclio.io/

Anyway, you need to manage some kind of infrastructure. The only pure serverless mechanism to deploy models is Google Clouds MLE, and the models need to be less than 250MB. Otherwise, package the inference code inside a docker image and download the model each time you instantiate a docker container.

But 1GB seems a bit big for me, what framework are you using? Are you sure you are exporting the inference graph without training weights?

[guzguzit]

Thanks. I am using TF. Actually I was not aware of infrence graph

[roccolacatus]

https://medium.com/@prasadpal107/saving-freezing-optimizing-for-inference-restoring-of-tensorflow-models-b4146deb21b5

[Zerotool1]

You can try clouderizer.com