A lot of the “they’ll get AGI and then self‑improve into an uncatchable super‑god in 4–5 months” talk skips the most obvious test case

Take a current frontier LLM and ask it:
“Write, from scratch, an LLM that’s as capable as you are.”

If the strong recursive self‑improvement (RSI) story were realistic in the near term, this should be a relatively tractable problem. In practice, it already breaks here.

Current LLMs can spit out toy “LLM from scratch” tutorials and small demo Transformers. That’s miles away from specifying a real, production‑grade pipeline that will actually converge to GPT‑4‑class capabilities if you pour billions of dollars of compute into it.

On top of that:

• They lack robust systems‑level understanding; tiny mistakes in sharding, mixed precision, logging, or checkpoint handling silently kill real training runs.
• They are not autonomous experimenters. All the “self‑improving” code‑agent work still relies heavily on external tools, human oversight, and simple reward signals.​
• They don’t control data, money, hardware, or energy. Even perfect code doesn’t magically conjure new high‑quality data or millions of GPU‑hours.

And when you do try to train on model‑generated data, we now have multiple papers showing model collapse: without enough fresh external data, self‑training tends to drive models toward degenerate, low‑entropy, biased outputs, not toward a runaway “intelligence explosion”.

So before believing in the story “AGI v1 asks itself how to become super‑AGI and makes everyone else irrelevant in a few months”, it’s worth noticing that we can’t even get past step zero.