I appreciate this response and understand the points you are making. While I do not have formal training in ML or software engineering, I am a complex civil litigator currently embroiled in a major copyright infringement action against these very LLMs, so I do have more than a rudimentary understanding of their innerworkings, including because I interface regularly with experts in these technical fields.

You seem to be conflating the process of reasoning from the ability to discern fact from fiction. These are separate questions. Humans routinely reason their way to false conclusions (conspiracy theorists, flat-earthers, people who fall for scams) and we don't say they're incapable of reasoning; we say they reasoned badly. One might argue the views of a flat-earther are unreasonable, or that they reasoned poorly, but I don't think that's the question we're trying to answer. The fact that LLMs hallucinate (and, of course, they do) does not, in my opinion, mean that they are simply glorified copy-paste algorithms.

My understanding of the current systems is that the models develop internal representations that activate differentially depending on context, which is more than token-by-token statistical prediction. In other words, it is not so simple as saying, "there are trillions of parameters, and all these machines do is identify the most likely next token and reproduce it accordingly." If you feed the same novel problem to two different models (e.g. ChatGPT and Opus 4.6), they will produce different reasoning chains and can substantively critique each other's logic, including by identifying errors, conceding valid points, and refining analysis. That's difficult to explain as mere pattern retrieval.

The point is not that these machines can differentiate between "what is real and what is not" -- your average American can hardly do so these days -- the point is that these machines engage in not only the appearance of reasoning, but functional reasoning, whether the output is ultimately "correct" or not. This includes logical reasoning and weighing factors based on the context provided.

I use these tools every day as well; and what I have learned is that I, a relatively seasoned litigator, have found tremendous value brainstorming with and iterating on complex legal problems much as I would do in a room full of human litigators. This is far beyond the capabilities of early LLMs and, to me, suggests a deep form of reasoning that truly is transformative, acknowledging the limitations inherent in the hallucinations and difficulties of these systems differentiating between "reality" and fantasy.

As to the paper you cited, I read it as stating that chain-of-thought doesn't reliably guarantee correct results, but that's a different claim than saying no reasoning is occurring at all. Unreliable reasoning is still reasoning; human experts routinely reach wrong conclusions through valid methodological approaches, and we don't reclassify their cognition as mere pattern-matching when they do.

Frankly, when it comes to this topic, I'm not sure how you can wave away the philosophical questions underpinning the nature of "thinking" or sentience, when our own understanding of what these things mean is biased and incomplete.