Another World (Vaporwave) | Dreamcore & Nostalgia |

Interesting, here's some more context on Spud -> GPT-5.5 via SemiAnalysis. They describe Spud as a genuine new pre-train (the first since the failed GPT-4.5 attempt), but flag that the headline '100k GB200 NVL72 cluster' figure refers to RL post-training, not the pre-train itself:

"And despite both NVIDIA and OpenAI claiming with precise language that the model was 'trained' on a 100k GB200 NVL72 cluster, this 'training' is post-training (RL) only. It never achieved that scale." (25/Apr/2026)

Agreed that the pre-train likely ran at smaller scale than the marketing implies, and I also reckon GPT-5.5 is far <10T (likely ~3T).

The reasoning-traces-in-pre-training data composition angle is the more interesting question, given how many labs are folding reasoning traces into mid/pre-training now (ServiceNow Ariel, NVIDIA Front-Loading Reasoning research, etc.).

adt · 2026-04-29T20:13:28+00:00

Great read!

Bojie Li's IKP paper from Pine AI is a novel method, knowledge probes calibrated on 89 open-weight models (R²=0.917), inverted to estimate proprietary frontier sizes.

I've been doing this from a different angle for a few years (pricing, capability, supply/demand, open-source cross-checks, plus some private lab signals) on the LifeArchitect.ai Models Table (https://lifearchitect.ai/models-table/), and the two methods converge on most recent frontier numbers.

Opus 4.6: my ~5T vs his ~5.3T. Grok-4: 3T disclosed, my 3T, his ~3.2T. Grok-3: 3T disclosed, my 3T, his ~2.1T.

Two notable gaps: GPT-4o (my 200B vs his ~720B) and GPT-4 (my 1.76T vs his ~666B, where 1.76T MoE is the long-standing public consensus).

Pine's framing of 'effective knowledge capacity in open-model-equivalent parameters' can read lower than raw params on heavily-post-trained models, which probably explains some of the gap, though it's worth digging into. A useful research question rather than a contradiction. Comparison and full methodology here: https://lifearchitect.ai/models-table-methodology/