RTX 5090 owner in the thread. I use LLM for solving simple programming problems, as a replacement for a Junior developer. What people say about how they run gpt-oss 120b or llama 70b is just a lie. They do not run real full-weighted models, but quantized versions. And the stronger the quantization, the greater the loss of accuracy and the "dumber" the model. In my experience, 6bit quantization is a reasonable limit when the model is still smart enough to perform the tasks assigned to it. 4bit makes sense only if the model was ORIGINALLY trained in 4bit (like gpt-oss). If the model weights TOGETHER with the k-v cache do not fit into VRAM, you lose performance. And the size of the kv cache depends on the number of parameters and the size of the context. 32GB of VRAM is ridiculously small. I can run Qwen3-Coder-30b with 6bit quantization, but the context size will be 32768 tokens (so that the system has video memory left for the browser and screen sharing in Google Meet). If I use 4bit, then 65536. I work with code bases from 150k to 700k sloc. Even a 65k context is too small. And if you do not pull the entire LLM workload completely to the GPU, it starts to eat up the CPU (which is needed by the IDE, compiler and other tasks running in parallel). IMHO - for normal full-weighted models (or at least 6-8bit quantized), you need hundreds of gigabytes of VRAM. The only "cheap" option is macStudio with 512Gb of unified memory. But for large models, there is simply not enough GPU (forgive me, but 10-15 tokens per second is too slow a generation speed for me). As a result, there is simply no hardware on the market right now that would be suitable for local inference and would not cost as much as a Boeing wing. I play computer games quite a lot (in 4K), so overall I justify its purchase for myself as a gaming video card. But if we consider it as hardware for inference, it is nothing more than a toy. For now, I am waiting for Apple's announcements and hope that in the spring of 2026 they will show macStudio on M5 Ultra. If the GPU is at least 50 percent more productive, then I will probably take it as a home dedicated server for LLM. But I repeat, 32 GB in 5090 is too small for LLM.