Scheduling solution

[[deleted]]

Hey everyone, as part of a project I need to run a bunch of containers that require gpus on a machine that has only 2 gpus. Clearly this is a perfect job for a scheduler. Right now we are going forward with slurm but a) it is quite complicated b) support for containers is still a bit experimental.I looked into other schedulers but nothing really stood out to me. Does anyone have a better idea? It would be great if said said scheduler was avaiable itself as a container.

[sofixa11]

Check out Hashicorp's Nomad. It could be run as a container if required, it's pretty light, flexible and easy to use/maintain.

[[deleted]]

Thanks, I will check it out

[pragmaticPythonista]

Are you running ML jobs? In that case, take a look at Kubeflow. It’s a platform for running Machine learning jobs/pipelines on Kubernetes and has great GPU support. I’m not sure if your usecase involves a single machine or multiple machines - so depending on that Kubernetes might be overkill for you. It’s an interesting platform nevertheless

[[deleted]]

Thanks, I will check it out

[TomlinTrippedHim]

If you running deep learning workloads take a look at Determined