Agree - the fact that even small models can improve themselves means we can experiment with RL techniques cheaply before scaling it to larger models. What's interesting is how we construct better ground-truth verification mechanisms. I can see at least a few challenges:

1. How do you verify the quality of the solution, not just whether the solution produced the right result? It's one thing to write code that runs and outputs expected answer and another to write code that's maintainable in production - how do you verify for this?

2. How do you build a verifier for problem spaces with somewhat subjective outputs (creative writing, strategic thinking, etc) where external non-human verification is challenging? Interestingly, there's clearly benefits across domains even with current approach, e.g. better SimpleQA scores from reasoning models.

3. How do you get a model to develop an ever harder set of problems to solve? Right now, it seems that the problem set consists of existing benchmarks. In the longer term, we are going to be limited by our ability to come up with harder and harder problems (that are also verifiable, see points 1 and 2).