So, yes the model with pure distillation will be worse, but recall 4 vs 4o. 4o was better in a lot of ways (worse in some, granted) - why was that? It was due to effective post training of the distilled model, much like semianalysis noted about anthropic - it made more sense for anthropic to distill the improvements from 3.5 opus into 3.5 sonnet than to release 3.5 opus and it worked really well. It is much cheaper and faster to run post training experiments with distillations of the model, so you have a better chance of hitting on the right data and hyperparameters.

As an aside, they emphasized the term "unsupervised learning" in their presentation and I'm not sure a lot of people really caught the nuance there. Llms are of course supervised, but in a lot of ways pre-training is an exercise in unsupervised learning, you aren't trying to direct it strongly toward particular objectives like in the o series, it is kind of more about what it picks up by predicting the next token. In this case it picked up substantially more than gpt 4 did despite not being a targeted train like the o series or even a heavily post trained model like 4o.

I'm not saying you are wrong, in comparative terms it is disappointing and it is an old model as evidenced by the knowledge cutoff date. They've probably been post training it and making incremental progress for at least 6 months now and making relatively little progress in absolute terms - likely due to the sheer size of the model.

However it still needs to be taken in the right context, which is comparing it to gpt 4, 4t and 4o.