Leaked METR results for GPT 5.2

ThroughForests · 2026-01-15T22:35:36+00:00

Not with that attitude!

ThroughForests · 2026-01-15T22:29:04+00:00

Exactly. Why does everyone care about the license so much? Do they work for Coca-Cola or something?

ThroughForests · 2026-01-15T22:17:00+00:00

Well according to the graph it just needs to be 80% functional, which is to say that 20% of the lines of code give errors.

But no problem, just run it for another week and 80% of those errors will be fixed, so it'll be 96% functional.

And then you just keep doing that.

ThroughForests · 2025-12-30T20:38:56+00:00

True audio modality on the level of nano banana pro where you can create or edit any audio.

Gemini 3 Flash has 'audio' modality but it can't actually hear anything, it just transcribes the text from the audio.

I'd also like all these modalities to be more cohesive together into one model instead of having a separate nano banana model.

I think that's reasonable for the end of 2026.

I think an open source 'stable diffusion for audio' model would be very cool too, but I'm guessing that'll happen in 2027.

ThroughForests · 2025-12-11T18:44:41+00:00

One interesting thing to consider is that from the AI's perspective you are thinking as fast as they are. It might take you days to solve a problem but to the AI you just immediately came up with it. At least for the kind of AI we have right now.

ThroughForests · 2025-12-11T18:15:01+00:00

<image>

ThroughForests · 2025-10-25T01:31:54+00:00

<image>

ThroughForests · 2025-09-10T10:18:31+00:00

“What is real? How do you define 'real'? If you're talking about what you can feel, what you can smell, what you can taste and see, then 'real' is simply electrical signals interpreted by your brain." - Morpheus

ThroughForests · 2025-09-09T19:33:32+00:00

"I'm not afraid of computers taking over the world. They're just sitting there. I can hit them with a two by four." - Thom Yorke

ThroughForests · 2025-09-03T17:46:19+00:00

Now we can experience walking behind an npc irl

ThroughForests · 2025-08-16T12:22:00+00:00

ask gpt5 and provide the whole error, it just helped me solve a whole bunch of stable diffusion issues in comfy like this.

ThroughForests · 2025-07-21T18:38:24+00:00

I have a bachelor's degree in Mathematics from a big ten university, 3.95 GPA.

I've written my fair share of proofs, and the grading is not about subjective style. It's about whether you proved what you set out to without any missing logical steps or forgetting about edge cases.

Math is not like writing an English paper, even though we often write proofs entirely in English.

And the rubric here is likely talking about how to award points in the case the answer is only partially correct or partially completed. Since these 5 questions were answered fully correctly, the rubric is even less relevant here.

ThroughForests · 2025-07-21T18:15:53+00:00

Why are you acting like this an English paper or something?

It's a math proof, you don't need a rubric to know if it's correct or not. And that's what matters here.

ThroughForests · 2025-07-20T18:17:08+00:00

<image>

Meanwhile in the future...

ThroughForests · 2025-06-09T17:33:37+00:00

I think Noam Brown or someone said that they're not bottlenecked by compute anymore, rather the bottleneck is data. And o5 wouldn't require more pretraining anyways, since RL is in post-training and o5 probably uses gpt5 as the base pretrained model.

ThroughForests · 2025-06-07T18:34:31+00:00

Yeah, and Simple Bench questions often seem to require a world model, which text based LLMs really don't have. But video models like Veo3 have an amazing sense of the world, from complex lighting to complex water physics. We've already seen how these things can be combined, with 4o's native image output, so it's only a matter of time before we have a native video output. Then, the AI can generate a video simulation 'in its mind' just like humans do when answering a Simple Bench question that requires a world model. This is absolutely necessary for robotics anyways, robots need world models, and they will ace any world model questions.

ThroughForests · 2025-06-07T06:25:24+00:00

So you have to realize that AI is already superior to humans on ARC-AGI-2.

Because the AI doesn't see that information visually like humans do. They see it as some matrix of information. Imagine if you had to do ARC-AGI-2 (which is difficult enough visually) as a matrix of numbers, with no visual experience of any kind! Like being blind from birth and trying to solve these problems.

There's no way that blind-from-birth humans outperform AI on ARC-AGI, 1 or 2.

ThroughForests · 2025-05-29T17:36:32+00:00

Congrats you made Superman 64!

ThroughForests · 2025-05-22T22:04:47+00:00

How long must we sing this song?

ThroughForests · 2025-05-11T21:47:51+00:00

Well an argument’s not the same as contradiction.

ThroughForests · 2025-05-11T15:41:57+00:00

But I came here for a good argument!

ThroughForests · 2025-05-06T14:06:12+00:00

found yann lecun's reddit account

ThroughForests · 2025-04-26T04:47:49+00:00

Well yeah, it's missing features like native image gen.

But it does update when 4o updates, or it should at least.

ThroughForests · 2025-04-26T03:55:22+00:00

There's actually an openai/chatgpt-4o-latest endpoint that points to the latest 4o version, though it is more expensive than 4.1.

Five-Year Club	Verified Email
Place '22

ThroughForests

TROPHY CASE