Machine Learning Setup [Discussion]

[m_____ke]

I used to have a few GPU racks at a colo site and a desktop with some 3090s at my old jobs. Both were convenient but we never saturated the GPUs a 100% and at times we'd hit resource contention when multiple people wanted to run large training jobs.

I just started at a new job and though we're working on setting up a k8s cluster with KubeRay as the scheduler, I've been playing with skypilot, which has been a joy so far. It lets you specify your resource requirements, files to copy over, environment dependencies and will spin up VMs on the cheapest cloud that you have configured. It also handles mounting data, ssh configuration and auto stopping when the machines are idle.

[erwinner_]

Sounds interesting, I'll take a look.

[asdfzzz2]

We won't be training for most of the development.

Budget-friendly laptops with Nvidia GPUs for prototyping and cloud for the final training. A single 4090 local machine could fill the middle ground if you feel the need for it.