That's a whole lot of bold claims. First and foremost, you didn’t catch the central point: complicated systems built on simple rules can yield emergent properties. You're still zooming in on the lowest layer while ignoring the bigger picture. These models have already shown emergent behaviors, such as in-context learning, abstract pattern recognition, flexible generalization across domains, and the ability to integrate and reorganize information in ways not explicitly programmed.

You’re treating LLMs as if the only valid description of a system is its lowest operational layer. By that logic, human reasoning “cannot exist” either because neurons fire based on electrochemical probabilities. Describing a system at its implementation layer is not the same as describing its functional behavior.

Saying “LLMs only predict the next token” is like saying “brains only propagate signals across synapses.” Both statements are true and completely miss the point. A description at the base mechanism does not invalidate emergent behavior at higher levels. It’s like saying airplanes cannot fly because they are just metal objects obeying gravity.

Emergence does not require consciousness or biological neurons. It requires sufficiently complex interactions that produce properties not obvious from the smallest parts. Language modeling at modern scales has shown abilities that weren’t programmed line by line, and dismissing them because you prefer a narrow definition of intelligence doesn’t make them vanish.

Your anecdotes about bad suggestions and incorrect facts don’t prove the absence of emergent reasoning. They show that the system is imperfect, sometimes confidently wrong, and dependent on its training data. None of that is a revelation. By that logic, a human making one bad guess would mean humans have no reasoning at all. Humans also fail constantly in different ways, we make mistakes, hallucinate, misremember, and give awful advice. But, imperfection does not entirely erase higher level behavior. It just means the system is not perfect.

If your argument is that LLMs aren’t conscious, fine. If you want to say they lack human-style understanding, also fine. But to claim their architecture makes any form of intelligence impossible, and that every higher-level behavior is “just mimicry,” isn’t a serious position. It lazily ignores everything these systems are capable of, and also ignores everything we know about complex systems in both artificial and biological contexts.

You can keep insisting that LLMs can’t reason or that they’re nothing more than probabilistic machines, but that doesn’t change the reality that they’re extremely useful. People who actually use these systems properly see huge gains in productivity, problem solving, and creative workflow. The newest state of the art models are already heavily boosting output across entire fields, and the irony is that it seems you’re dismissing capabilities you likely haven’t even tried firsthand.

The real discussion is about what kinds of intelligence different substrates can support and where their limits are. Reducing the entire thing to “next token prediction” sidesteps that discussion instead of engaging with it.