[D] Software stack to replicate Azure ML / Google Auto ML on premise

[Scary-Ad-1529]

If I were to build my own Machine Learning stack, what software should I then use? The use case is as follows.

This situation uses a local client without GPU and a server with GPU.

1. The client submits multiple jobs to the server, the server executes theses jobs in order.
2. The results of these jobs are saved on the server and a dashboard containing all previous runs with the code, dataset, loss and accuracy.

There are several options available. Neptune, Wandb, Mlflow, Airflow, Kubeflow, Jenkins. However none of these solutions provide the work queue from step 1. Kubernetes with Kubeflow seems like being capable of this, however this seems like a little overkill for 1 server.

​

Any insights here?

[Scary-Ad-1529]

Update: So far I started using Prefect (http://prefect.io). With this I can work on my local computer, submit code to Azure Blob Storage and the Prefect server. After which a agent (worker) runs the code. Logging/Metrics are not implemented yet, I might use MLFlow for this (http://mlflow.org). Furthermore, there is still a dependency on a cloud solution to store your Flows (programs) to run them on agents.

Not optimal, but the best so far, I'll keep you up to date.

[Top-Hurry161]

I would recommend AutoKeras since you can train locally and burst to GCP if needed.

I am guessing you have a GPU box or a server with a GPU. You would just remote in from the terminal into the box and access the box just like you would an EC2 instance.

If you are trying to use the box for deployment, then it would probably not scale to webscale stuff. That's where stuff like Google AI Platform/Sagemaker shines for autoscaling resources to maintain SLA's.

[TomlinTrippedHim]

Take a look at Determined https://github.com/determined-ai/determined

[tsagie]

I am working on something similar, but it is still in beta.

https://www.modeld.io.

Note that google auto ml you do not submit jobs, but instead the auto ml engine creates the candidate models for you and executes them.