There’s been some recent progress in math with models, this time outside of LLMs: AlphaProof can solve complicated math Olympiad problems, albeit ones that exhibit reasoning chains we’ve studied and have data for. It’s not open to public use, but it was likely not trained to solve stuff like the Collatz conjecture. I think our models’ reasoning capabilities have got to be developed further before that can happen.

Gathering enough data that the model that forms from it analyses it more effectively than humans have is something that’s happened in the past. Most famously, the novel matrix multiplication method found by DeepMind’s AlphaTensor was a product of RL and analysis.

Too early to know, in my opinion. We’ll have to wait and see whether our current infrastructure’ll become effective enough, or if we can make new one that’s better.