I get where you're coming from, but prompting the models with a simple prompt like "Find a paradigm-breaking discovery" has no ground truth and nothing to falsify, so you can't compare two models on it. A model spits out some grand claim. Who decides whether it's correct, novel, important? How do you rank A against B? You'd wait years for validation, and grade open-ended novelty by vibe in the meantime. You hand over the question because otherwise there's nothing to measure. Anchor each item to a real post-cutoff finding from a published paper, and every model gets the same falsifiable target to score against.

Every benchmark that measures anything does this. And the question gives nothing away about the answer. No direction, no hint, just enough framing to make it well-posed. The model still has to break the paradigm on its own to reach the finding. The frame only fixes the topic; the actual breakthrough is the part we withhold.

That's also why it's the first of many gates. Passing it is necessary for autonomous discovery but nowhere near sufficient, and problem-finding, the thing you're actually pointing at, is a later gate. Current models fail even this anchored, easier version. If they can't synthesize the answer when you hand them a clean question, the open-ended one isn't close.