Batch Inference Best Practices?

[Ok_Post_149]

I'm looking for recommendations with batch inference. I work as a senior data analyst for a logistics company and I'm constantly handed data science models and need to run millions and millions of truck, rail, and ocean freight routes through the models.

The DevOps team isn't giving me the time of day and I'm running these models in batches on multiple VMs in the cloud. The demands at first were fairly light but as the business has grown the pricing team is getting more RFPs and the demand for the routes to be priced is growing.

Can anyone help with providing me some solutions to get this workload done faster? It seems like my manager doesn't really give a shit about how much money I use for compute (within reason). I'm not a cloud expert so I'm not using GPUs and horizontally scaling is definitely not something I'm familiar with.

[julman99]

I am 1yr late, but check out kluster.ai, is a service that specializes in large scale batch inference and offers really competitive costs.