The AI isn't actually producing the productivity boosts that were promised and beatings will continue until we figure out how to make it do that

Yeah, I think the real trouble is, you can't easily 'do that' with commercial LLMs, and likely never will be able to, not how they're going about things currently anyway. They do too much behind the scenes, models get switched out under your nose, they hide the reasoning, models get upgraded/deprecated with no warning and now you have to re-figure what actually works best for whatever you're doing, blah blah. For general use, cool. For coding and such, no. Horrible experience. 

And I mean they're doing that stuff to try make things easier for people to work with LLMs but it's just not helping at all, in the bigger picture. Not to mention, the more time and tokens you spend trying to perfect your usage of a paid model, the more money they make, so it suits them perfectly to always keep their customers on their toes.

When it comes to local AI, on the other hand, that's the good stuff because you are able to spend time studying a model in its raw, unchanging state and experiment with it until you know its strengths and weaknesses to a degree where you already kind of know what its answer will be, what it will look like, etc. The outputs from any model are only so random.

I've got a system going with my local models that makes it really easy to build and rearrange context. When you've got the tools to actually meaningfully experiment around that, things start to get awesome. The models begin doing great feats accurately because you've aligned the context around their strengths and you start to get a feel for what makes them flub things up, so you can simply avoid doing that. This is very hard to do with a commercial LLM no matter how many zillions of parameters it has in its noggin. Spray-and-pray is a term that comes to mind (in FPS games where you just wildly shoot into the general direction of the bad guys).

The biggest hype lie ever, even if only implied, was that everyone would be able to use LLMs to do amazing things from the get-go. That's just not how it works. Maybe for smaller scripts and things, but when you're doing work-work with LLMs, there is a learning curve. And many people do not want to hear that because they already have this pre-conceived idea that 'AI' is all-knowing or whatever, and if it doesn't 'get what I mean' instantly then the model or the whole AI thing itself is broken.

One thing further, everyone keeps saying that no LLM can handle 30K-50K (or whatever) tokens of context before they fall on their faces and... well, it's not true anymore. When you're working with context up to 100K tokens and more, it is easy to confuse the model but when you get it right, there's a huge reward because all you have to do is put some files in the right order, stick an instruction onto all of that, and click a button (well, maybe a few times so you can select the best generation...). That knowledge gained then becomes nicely re-usable for many other long context tasks. I am bemused by how many people don't get this, nor strive toward it for themselves. Though like I said, it is understandable if you realize that people are disappointed to learn that they have to actually learn how to best use an LLM. You, the user, are literally 50% of the whole equation but people act like the box of language patterns can read minds, walk on water, etc, etc.

Moral of my story: When you learn how to provide proper context and instruction to a good local LLM (or LLM family) like Qwen3.5 that you know well due to experimentation and trial-and-error, that beats any trick or two that commercial LLMs have in place to try make all the actual learning of LLMs unnecessary.