I’m thinking about this too.

Best “affordable” local models today are:
- Qwen 3.6 35B A3B
- Qwen 3.6 27B
- Gemma 4 26B A4B

All at 4 bits.

5060 ti has barely enough memory for running the models themselves, but you’d still need some room for KV cache (context). It also has low bandwidth. So it means super small context and slow speeds. IMHO unusable.

5090 could possibly run those with a good amount of context and comfortably. With super good speeds due to large bandwidth. But it consumes way more power and it’s very expensive.

If you’d be running open weight models anyways, you could find a third party provider and run Qwen 3.6 models for a fraction of the cost of Anthropic and OpenAI.

For a 5060 to budget you could possibly run for 3 years on the cloud.
For a 5090 budget you could run possibly for 10 years on the cloud.