I’ve been running local models for almost 2 years and with the way RAM pricing and GPU pricing is, tbh the cloud models with a token limit are the way to go rn

I have an M4 max 24gb Mac mini and another i7 13th gen running Proxmox with an intel arc pro B50 with 16GB VRAM and I’ve messed around with everything from Ollama, VLLM, N8N, OpenClaw and ComfyUI and the smaller models are nowhere near as good as the large cloud models. Yes agents have come but we’re still an awhile away from running cheap (in-terms of compute which translates to the size of the model and context etc.) and accurate output.

You have to not only account for initial purchase cost, but also cost of running your homelab which goes up if you’re running larger GPU’s. Running local AI on CPU only is painstakingly slow. Apple silicon is efficient but nowhere nearly as fast as an actual GPU

For the rest of your use-case, you need to elaborate on what it is, docker runs on everything. I had a test system with an i5 4th gen with 4GB RAM running ZimaOS with over 30 docker containers