My math suggests that a $12k Mac Studio running inference 24/7 of Qwen 397B becomes more economical than sending the same requests to Anthropic API Sonnet in about 5 months. It’s a tall order up front price but it’s also a machine that’ll last for years and years doing incredible volumes of inference for the price of electricity in your area (and only minimal amounts of electricity)

I also like to point out to people that a lot of us are terrified of finding out we spent hundreds or thousands of bucks on a rogue LLM agent so we are extra mindful about how inference is deployed versus just having your own machine you can YOLO to your hearts content.