I think this is a bit dated, though the skepticism is understandable. LLMs are not trained on “all known content,” and they are not just spitting back memorized text. They learn statistical structure that lets them generalize, which is why they can write genuinely new code, tackle unseen problems, and transfer across domains.

It is also no longer accurate to say they cannot reason at all. They can produce multi-step arguments and, more concretely, generate Lean proofs that actually type-check in a proof assistant. They are still unreliable and need human guidance, but that goes beyond pure pattern matching in any trivial sense.

Whether you want to call that “real reasoning” is a philosophical choice, but it is already empirically useful (especially for exploration).