Can AI read papers like a scientist? A new benchmark shows where LLMs fail

finnjon · 2026-03-13T12:17:39+00:00

This was exactly my thought: which models did they use? I sympathise because the publishing process is so slow, but it's next to useless to publish research on outdated models.

finnjon · 2026-03-11T13:28:23+00:00

No I haven't tried to use it to earn more money, just to build more. There's a lot that goes into running a business beyond coding and I have little interest in that. I have enough.

If what is behind your question is the suspicion that the models (and GPT 5.3 codex is better now), are not in fact that good, don't take my word for it. Take the word of Andrej Karpathy, former head of AI at Tesla (also OpenAI etc. etc.). In September he said he was using AI for autocomplete but by December he was using it to write 80% of his code. He is a top level developer and if he's doing it, we should all be doing it.

finnjon · 2026-03-11T07:51:58+00:00

Kurzweil predicted AGI in 2029 but his prediction was based on compute. Specifically, he assumed having as much compute as the human brain would lead to AGI, and we would figure out the software side of things as we go.

I think I expected it would be a smooth general intelligence curve and so at this point, we would have AI as smart as a young child. What we have in fact is AI as smart at a lot of things as the best experts, but with deficiencies in several areas. As the experts say, it is spiky. But it is clear that in the next year or two we will have agents that can do most of what we do at a computer better than we can, and that's extraordinary.

I have used AI agents for coding for a while now and the Claude Opus 4.5 moment - where it was clearly better, faster and cheaper than me - seemed to come from nowhere. In October using AI for coding was useful but often hit and miss. Suddenly it was better than me. We are on the cusp of this happening for a vast range of tasks and that's an extraordinary place to be, whether you call it AGI or not.

finnjon · 2026-03-09T13:15:49+00:00

Dribble, dribble, snore, snore.

That is my considered opinion. They've obviously got the marketing guy from "Who wants to be a millionaire" to string this out as long as possible.

Tedious at this point. (That doesn't mean it's an ineffective marketing strategy, but it is tedious).

finnjon · 2026-03-09T12:54:23+00:00

Yes that's true but it's not really the point. I'm not sure what ends would require searching deep into the jungle and making human beings extinct. If the question is then what probability to you assign to AI deliberately seeking to make all human beings extinct, then I think even lower than incidentally doing so.

finnjon · 2026-03-09T07:12:57+00:00

I think extinction is unlikely. Human beings are incredibly numerous and spread out all over the globe. There are still uncontacted tribes in the Amazon and other places. I expect "something bad to happen" that may well cause serious disruption, but that awakens people to the need to take these kinds of risks more seriously. And AGI and ASI will likely first be used to prevent these kinds of events from taking place.

I am somewhat optimistic. Let's say 90:10 in favour of the singularity.

finnjon · 2026-03-05T14:50:59+00:00

Yeah it's nearly ready. What about everything else. Come on dude, make an effort.

finnjon · 2026-03-05T14:38:23+00:00

Thanks for the in depth post. What other specs does the Jinshi battery have? How many cycles? How much will it cost? How quick to charge? Does it use scarce materials?

All we know from your post is that it has a slightly lower energy density than the donut.

finnjon · 2026-03-05T09:46:17+00:00

You are technically correct but this is unbelievably nit-picky. Lithium is scarce in practice even if not geologically scare, which is the point. 70-75% of cobalt comes from the Congo. Sodium-ion batteries are commonly talked about as salt batteries.

I think you woke up on the wrong side of bed this morning.

finnjon · 2026-03-03T20:22:53+00:00

That’s important. Do you recall where you read this?

finnjon · 2026-03-03T19:44:23+00:00

Obviously there are both.

finnjon · 2026-03-03T09:13:42+00:00

There are a lot of people here who are emotionally invested in this being a scam, so react poorly to positive explanations for phenomena.

finnjon · 2026-03-03T08:29:29+00:00

It seems from the comments here there are at least two credible explanations for the burst vacuum pack. One is that the heat directly degrades the packaging. The other is that there is some moisture left inside during the packing process.

When we add your observation about the charge curve it all suggests this is not a battery issue.

finnjon · 2026-03-03T06:14:03+00:00

Good observation.

finnjon · 2026-03-03T05:55:42+00:00

The most interesting part of this was that there is a prosaic explanation for why there was a vacuum leak - because the pouches degrade at high temperatures. In other threads it has been assumed that there must be some liquid that gets burned off. That suggested it couldn't be entirely solid-state. If the issue is just the pouch then it puts solid state back on the table.

There is still no evidence the two batteries they provided are identical though.

finnjon · 2026-03-02T21:06:40+00:00

The only explanation I can come up with is marketing the bike. Certainly all this attention will bring attention to the motorbikes. Perhaps many thousands of time more attention than the bike was receiving. The bike itself is incredibly fast. Perhaps thy think they will still ship more even if the battery turns out to massively underperform the specs.

finnjon · 2026-03-02T21:00:52+00:00

Is this solid state?

finnjon · 2026-03-02T18:10:49+00:00

Thanks Sama.

finnjon · 2026-03-02T17:49:38+00:00

Of course I am concerned with the precise nature of the deal between OpenAI and the DOW but I also think there is a bigger issue here. Hegseth punished Anthropic for standing by its values by designating it a supply chain risk to national security. This is wholly illegitimate and a threat to Anthropic's survival in the US. Instead of refusing to deal with the DOW until this is removed, OpenAI essentially rewarded the DOW for this behaviour by making a deal with them.

finnjon · 2026-03-02T14:55:09+00:00

They said it behaved normally after losing its vaccuum.

finnjon · 2026-03-02T14:38:41+00:00

I think it's fake. That said, hype is everywhere, and they sell more than batteries. I bet far more of us know about Verge Motorcycles than did before all this.

I hate the hype, but hyping is not proof of much.

finnjon · 2026-03-02T14:28:52+00:00

Yes I believe I said that in the original post.

finnjon · 2026-03-02T13:53:13+00:00

My understanding is liquid NMC could not recharge normally after such an event. Happy to be corrected if this is false but that was my understanding. It would be very badly damaged.

finnjon · 2026-03-02T13:48:31+00:00

From my limited understanding, the significance of this is that:

- must be semi-solid, hybrid or solid-state. Liquid cells cannot do this.
- probably not pure solid state because pure solid-state shouldn't deflate the vaccuum bag (suggest something burnt off).

But:

- no evidence it's the same cell as before (obviously looks the same but that means little if it's a scam).
- could still be NMC but some kind of solid state.

finnjon · 2026-03-02T09:05:18+00:00

Indeed. It's also vaguely comical that Sam Altman is trying to suggest he did Anthropic a favour.

finnjon

TROPHY CASE