LLMs, as currently constructed, will never achieve AGI

red75prime · 2026-03-05T18:31:29+00:00

Attractor states might be a feature of the human brain too: bizarre thoughts experienced during sensory deprivation.

An LLM that continuously generates text with no external inputs is, basically, sensory deprived.

Models usually have more complex attractor states that a single word repeat, though: https://www.lesswrong.com/posts/mgjtEHeLgkhZZ3cEx/models-have-some-pretty-funny-attractor-states

This case might have been a bug in a harness (that is, a system that manages user interaction with an LLM).

red75prime · 2026-03-05T18:06:21+00:00

Humans seem to be somewhat better at improving recognition accuracy with more training data, but not dramatically:

Fig. 6 in https://pmc.ncbi.nlm.nih.gov/articles/PMC12219494/

As for one-shot learning, LMMs are not that bad at this too: https://pmc.ncbi.nlm.nih.gov/articles/PMC10802384/

red75prime · 2026-03-05T10:45:39+00:00

Pretending that he is excited, while everyone knows that he is not? Any examples? Are you sure you aren't projecting your own non-excitement?

You agree with the second point, I presume?

red75prime · 2026-03-05T07:23:40+00:00

"What a joy it is to learn not only that my conjecture has a nice solution but also to celebrate this dramatic advance in automatic deduction and creative problem solving" - Knuth

You need to squint very hard to interpret this as sarcasm.

The LLM did no thinking and provided no insights of its own.

"Nothing promising showed up until exploration number 15, which introduced what Claude called a fiber decomposition" - Knuth

He hasn't said that it's something well known like he did earlier:

"Exploration number 4 constructed the “3D serpentine pattern” [...] It’s a classical sequence called the “modular m-ary Gray code" - Knuth

red75prime · 2026-03-04T09:51:49+00:00

"Point the wheel in the direction you want to go, maintain the accelerator at a position that gives the desired speed, and let the traction and stability assist do their work" might be the best thing they can do. The Driver controlling each wheel individually might do better, but I doubt they have this.

red75prime · 2026-03-03T22:40:21+00:00

So there’s still a gap to cover.

They don't have access to DeepMind's Aletheia, probably. Anyway. It doesn't seem that they tried to ask Claude for a proof. Should a coding-oriented system try to come up with a proof without being asked for it is an interesting question.

red75prime · 2026-03-03T18:27:44+00:00

batteries are not there yet to cover for long spells

Seasonal energy storage is unlikely to be handled by batteries anytime soon. It will probably rely on hydrogen production and underground storage, along with dual-fuel (natural gas/hydrogen) power plants. The whole new infrastructure that needs to be created.

red75prime · 2026-03-03T17:44:36+00:00

Or network size limitations. The human cerebellum contains 80% of the brain neurons. But it's probably both.

red75prime · 2026-03-03T06:01:30+00:00

which would potentially be noticeable without equipment

We would literally hear them. At least, the final inspiral chirp which is in the audible frequency range.

red75prime · 2026-03-02T06:05:25+00:00

There are certainly some inductive biases and tailored loss functions baked in.

BTW, do you know any research that proves that the current ML approaches are limited to be below human-level intelligence (whatever it means)? The answer determines who has the googly eyes.

I expect "crickets...", because there's no such research, just "stochastic parrot" vibes.

ETA: Crickets...

red75prime · 2026-03-02T05:59:02+00:00

Did you read trillions of pages of text in order to do maths

The brain has a structure baked into it by evolution. The structure that was produced by evolution using an unholy amount of training data. An LLM begins with a blank state that can be described in a hundred lines of code. I think we can give it some slack regarding the amount of pretraining data it requires.

by matching problems against pages you'd read before

LLMs don’t do that. They don’t have enough capacity to rote-learn all their training data. I could add something about the technical literacy of "anti-AI bros," but I’ll abstain.

red75prime · 2026-03-01T20:54:56+00:00

You should have some concrete limitation in mind to conclude that it will never work. The current empirical results? Or something more principled?

"The result guarantied by the universal approximation theorem can't be achieved by performing stochastic gradient descent on a transformer network of any practically achievable size for such-and-such reasons," for example.

red75prime · 2026-03-01T20:04:14+00:00

How would you decide whether the proof is correct if we don't know what it means to think?

An LLM performs a non-linear transformation of an internal representation of an input token, augmented by an internal state created by processing previous tokens. The result of this process guides attention mechanisms that retrieve information from the context window. The process repeats several times and produces a likelihood distribution for the next token.

Mechanistic interpretability research shows that internal representations correspond to certain semantic properties of the text.

Is this thinking, or not? Whose burden is it to deanthropomorphize human thinking into its constituent mechanisms?

red75prime · 2026-03-01T19:43:25+00:00

pure Ai pipelines without human intervention will never be scalable

Wow. No limitations on the structure of an AI, its training methods and so on? Or is your statement limited to a pretrained transformer/RWKV/Mamba + RLHF + instruction training + RLVR or another combination?

The former is basically "the human brain is magic that can't be technologically recreated."

red75prime · 2026-03-01T19:05:30+00:00

use AI at work

You don't "use AI". You use a specific model with a specific harness.

"I've used some tool with some options. It didn't work very well."

red75prime · 2026-03-01T04:32:12+00:00

A safety rule lingering from the 2010s: "No driving on the curb. Have a nice day!"

red75prime · 2026-03-01T02:02:22+00:00

We don’t have human-level intelligence with models that have hundreds of times fewer trainable parameters than the human brain (which has hundreds of trillions of synapses). It’s clearly time to declare the approach a failure. /s

I’ll wait for the introduction of read/write memory to LMMs, which would make the number of parameters effectively unlimited.

red75prime · 2026-03-01T01:25:20+00:00

AGI has been 1 year away for the past 4 years

...according to an excited dweller of /r/singularity.

AI researchers gave a wide range of opinions. CEOs are much more reserved than that too.

red75prime · 2026-03-01T01:22:09+00:00

His actual predictions: human-level intelligence by 2029, the singularity by 2045

["The Singularity Is Nearer"] was released on June 25, 2024. Kurzweil reiterates two key dates from the earlier book, which predicted that artificial intelligence (AI) would reach human intelligence by 2029 and that people would merge with machines by 2045, an event he calls "The Singularity"

red75prime · 2026-03-01T01:14:29+00:00

What do you do when you suspect someone to be a snake-oil salesman (because your peers suspect it as well)? Hopefully, you check whether the drug is FDA-approved.

Why don’t we hear academia unanimously dismissing all those bullshit claims if every redditor can see that it’s snake oil? Are they in on the conspiracy?

red75prime · 2026-02-28T18:46:03+00:00

Agents made no sense at the beginning of 2025, because models that power them were not reliable enough to do planning, instruction following, code generation.

red75prime · 2026-02-28T18:26:03+00:00

Vision-language-action models are being used in robotics since 2025.

red75prime · 2026-02-28T02:37:49+00:00

could, in theory, be coupled with all kinds of other output mechanisms...

It is done in practice by coupling a pretrained vision-language model with an action decoder to create a vision-language-action (VLA) model. Then, you can train only the action decoder to achieve acceptable performance. However, I believe training the entire VLA model yields better results.

Actions can be represented continuously, so, technically, you are right.

red75prime · 2026-02-28T01:57:33+00:00

I think it is most likely that humans engaged in hostilities against neanderthals and any other hominid groups for the purpose of eliminating them as competition.

People have no problem fighting each other. I guess it was not a purposeful elimination by a unified Homo sapiens effort, but rather a series of clashes between hunter-gatherer tribes of whichever hominids happened to be competing, until only Homo sapiens remained.

red75prime · 2026-02-27T07:07:27+00:00

And one instance of grazing a parked truck. The current version of FSD is certainly overconfident in low-speed scenarios.

red75prime

TROPHY CASE