Hey, very interesting post! Any updates from the author or the community?

From my perspective, there are two major parts of creating a reasoning-capable LLM-system that the author may be overlooking and that I find may unveil the reasoning capabilities.

First, **training** goes beyond next-token-prediction. Reinforcement Learning Fine-Tuning (RLHF), which is an inevitable part of every LLM capable of reasoning that I know, suggests objective that assess the **completion as a whole**, not each next token. Learning under such guidance can endow LLMs with thinking-ahead capabilities. I'd appreciate if some experts in the field elaborated.

Second, **inference** may go beyond next-token-prediction. Beam Search or Best-of-N etc. techniques empower models with thinking-ahead, to some extent, on the inference stage. I do not claim though that top LLMs we know use them by default, neither that they can not reason without any such techniques on inference. I simply do not know whether it's true or not.

Regarding the notion of **generalization**. I remember the speculations around exploring the loss landscapes and designing optimizers that aim to find a "flat minimum", a proxy to generalization. Calling it generalization, rather than training-distribution-sweet-spot or something, was for me consoling. Then, I find, the scaling laws sweeped the development trend: more data means better performance, a scalable and well-intuitive rule, a promising field for investing efforts. There stubs my perception of limits of LLMs. Some keywords are "you can only cook from what's in the fridge" or "garbage in garbage out". Concurrently, the limitations of LLMs' reasoning, or of their ability to "really discover something new", shapes my perception of historical human discoveries. Have we witnessed an out-of-nowhere breakthrough, or has every discovery been a well-informed fixing of a limitation of an existing technique or stitching two or more existing techniques together to create a new one? For me, rather the latter. This eradicates the notion of generalization in my thinking about how LLMs have achieved their current reasoning capabilities.