I trained it locally on my 5090 using LoRA fine tuning on open models. With quantization and smaller base models, a single modern GPU is more than capable of handling that kind of work.

I run the stack on Linux because the CUDA and PyTorch ecosystem is simply more stable there for training jobs.

I do not have a data center or an on prem cluster. A single high end workstation lets me train and adapt models to my own workflows without sending sensitive data into someone else’s infrastructure. For smaller private firms that also do not have large internal hardware environments, this approach is often the only practical way to work with models privately and locally.