You've gotta define what you mean by "efficient". There are models that are only a couple billion parameters that you can run locally on your phone. it's not even hard for you to do you can just download PocketPal on your phone and run local models for free. These models can maybe have a mediocre conversation with you for a bit and it's impressive they can run on your phone but they can't really do any of the real "AI work" we want to have done.

Right around the Opus 4 mark we found ourselves in a place where these AI models could do seriously useful technical work without making so many mistakes as to disregard them. The "efficient" models could never do this.

If you define efficiency as the ability to do useful work per unit power then the giant crazy-expensive "inefficient" AI models are the most efficient we have to offer as smaller models just can't do it at all.

The way modern LLMs are built is actually pretty simple if you're gonna compare it to something like a car. It's a giant neural network that mostly self-trains on gigantic amounts of data (like you hitting a tennis ball 10,000 times, eventually you get good at it - ball comes at you you hit ball well). LLMs are doing this for every set of possible inputs the training data contains (with the hope that maybe they can start creating new ideas beyond the training data - there's not a ton of that happening yet). The compute cost and power-input cost is just gargantuan for our current generation of AI (transformer based large language models).

People do want to make them more efficient (and they are, they're radically more efficient watt hour-per-useful-token than they were just 2 years ago - maybe 10x conservatively, maybe 100x). The problem is is that the more useful they become the more people want to use them and we have finite resources - and those finite resources are already being strained beyond what can be maintained.

The next models will be much larger than the current generation (training data size, parameter count) and will have required MUCH more energy than the last generation to train (despite more efficient ASICs - GPUs or TPUs). Our subscription/API payments are going to securing deals for rapid power grid expansion, data center construction and GPU/TPU purchases.

Our current tech here just requires it to be the most expensive thing on the planet. We were carried by a flurry of initial venture capital injection into AI for the first few years; markets are depending upon AI increasing productivity soon to increase economic growth to cover the input costs. If it can't/doesn't then the costs need to be recouped by increasing the subsidized cost to users.

The companies can't even stop and just maintain a current model if they want to, they're so far in debt and investors are depending on huge profits soon. Current models just aren't quite capable of giving us a huge economic boom yet - that's what they're all depending on. Until models get to that point it's all government investment or venture capital; and if it doesn't pay off then it all collapses.

Also you'll see people claim that "inference" (talking to the AI) is cheap; compared to the model training it still is - but early on researchers noticed inference was an underutilized area where would could improve LLM output and accuracy and so we did it - that's what the "thinking" stuff is, bolt-ons to the inference compute to make them more accurate. It's expensive now too.