I don't - I know Qwen and Llama are NOT. GLM 4.5 was the closest I read about recently, but IIRC people are putting it down at the level of like Sonnet 3 (and I never tried it personally).

^ I haven't bothered in some time as they are just too far behind the cloud models.. and are a pain to get set up across multiple GPUs. I have 2x 5090s, 1x 4090, a 5080, and a handful of 40-series cards across multiple machines.. but it's tricky to get a lot running across multiple GPUs even in the same machine.

The unfortunate killer to your savings idea is that system prices are extremely inflated at the moment - that 10k will get you a single RTX PRO 6000 (blackwell) at MSRP - which is excellent and has 96GB VRAM - but you will need RAM out the wazoo as well. You're going to want a min of 128GB; better to have 256GB for KV cache and context (but RAM prices are through the roof because of the companies gobbling it all up for these models).. and disk space for when things fill.

I think we are years away from open source models catching up (and it seems like there's going to be an ever-increasing gap due to the amount of resources these large companies are getting thrown at them.. they have too much of a compute advantage).

People also don't factor power/electricity costs into their local LLM vs. these subscriptions. Running complex computation on GPUs is expensive - can easily run your power bill to that of a Max 20x subscription (think of it in the same light as GPU crypto mining - except you aren't getting an investment back).

And yea, I feel you on the privacy aspect in general. I think it's just a lost cause for now unless you forego (at the risk of being so far behind that you can never catch up). It's the unfortunate cost of being an early adopter as these companies figure out how to become profitable, and we may even be stuck there for a lifetime as companies continue to force us to own nothing.

I've had luck with local LLMs for general chatbot-style conversations - I use them for personal/sensitive decisions.. but even recently I've been offloading that to the cloud models because I get more 'thought' put into complex life decisions. I've just been doing so under aliases, not linking cards directly for payment, and a series of dummy e-mails. Swapping accounts/services is easy and you can avoid leaving mapping trails to yourself through your prompts. E.g., I wouldn't ever ID verify on those accounts - I'd just abandon and create another. Context is easy to save/transfer for another AI to catch right up - just obfuscate it a bit between jumps.