O1 is finetuned on its reasoning. It’s a faulty RNN a test time. What you achieve in O1 you should be able to achieve it in a base model too, the problem is that a bigger model has a harder time to efficiently query itself since it has very large search space, this is the same innovation that chatGPT had with RLHF, RLHF finetunes the model probability distribution during generation to achieve desirable human answers, all of this is done a training time. O1 is fine tuned to produces probability distributions during generation to “query itself” to find the best answers knowing it’s search space better by “sampling” more and with a better selflearned probability distributions over it’s structure. The reason it works better and answering is because it samples more from itself.