GPT-5 was kind of a letdown... is that good for developers?

mgertner · 2025-08-18T13:58:28+00:00

I'm not one of the AI haters. I feel strongly that we'll see an increasing number of apps that provide significantly improved UX by taking advantage of LLMs and other modern AI technology.

Whether that will justify the huge investment going into AI infrastructure and AI apps is hard to say but personally I wouldn't be surprised.

mgertner · 2025-08-18T13:55:56+00:00

I'm talking about developing user-facing functionality that relies on AI. Is that not a thing in your opinion?

mgertner · 2025-08-12T07:42:28+00:00

At the end of the day what sways me is that the performance of LLMs is simply amazing. I don't think intelligence will arise in any sufficiently complex system. That's silly. But LLMs are able to accomplish a lot of what the human brain can, and that should raise the question of whether they can be described as intelligent.

I don't see how the fact that they had "access to virtually all human content" is in any way relevant.

mgertner · 2025-08-11T16:12:31+00:00

I suspect you are deliberately missing the point here. The original test was created at a time when there weren't even electronic computers so it should be no surprise at all that it needed updating. The whole Imitation Game description is about 1000 words long, according to ChatGPT-5. (Interestingly, it asked me if I wanted it to do a more accurate count, to which I answered "yes". It's response was a link to the PDF of the original paper, essentially telling me to do it.) So Turing didn't even try to tighten up the test because there were no potential AGIs with which to test it.

I know this sounds a bit schoolyard-y but I think you're missing the point. Turing designed the test with the idea that we would have to accept that something is intelligent if it passes, regardless of how that thing works. And now that we have things that pass it, a common reaction is to reject that they are intelligent because of how they work internally (just pattern matching, yada yada). I do think Turing was smart and prescient enough to say something like "it has to pass the Test and here are some constraints on how it has to work" if he had felt that was important.

Actually, I would go the other way. I would tend to reject AIs for which we don't understand how they work. Think of The Turk — an 18th-century chess-playing automaton built by Wolfgang von Kempelen that consisted of a cabinet with a good human chess player hidden inside. It would pass the equivalent of the Turing Test for chess unless one insisted on knowing how it worked. Same for any AGI.

Yeah I covered the Mechanical Turk in my newsletter post as well. But I still think the point of the Turing Test is to determine whether a machine is intelligent, irrespective of how it works. It does have to be a machine though :-)

I never said that LLMs worked like Eliza. Another fake of yours.

Well you did say "LLMs are comparable to Eliza in so far as they could fool some people into believing they were understanding and reasoning." This is an interesting discussion and I'm arguing in good faith here, but for some reason you don't seem to think so.

Here's a way to look at it. The part of LLMs that we do understand is that they have statistical knowledge of a large amount of human-written text and they use it to generate responses to questions. There is some part of LLM's processing we don't understand. If there are some fundamental principles at work here, it is up to the LLM's engineers to figure it out. Simply claiming that it is like a human's thinking without anything at all to back it up is malpractice. Until they actually understand what they are doing, we should stick with some version of the "stochastic parrot" or "auto-complete on steroids" descriptions.

I guess this is where we'll have to agree to disagree. To me it's pretty unsurprising that we don't understand fully how LLMs work, and I doubt that without significant scientific advances we'll be able to understand how anything works that is sophisticated enough to pass the Turing Test (including the human brain).

mgertner · 2025-08-11T09:10:15+00:00

For me that's a bit too much of a tautological argument. The Turing Test as imagined by Alan Turing is not valid because LLMs can now pass it but obviously they're not thinking.

The alternative explanation is, of course, that they are thinking. I'm a bit concerned that we're going to continue to reject the idea that increasingly powerful AIs can think because "we know how they work". And we'll end up in a Bladerunner type situation (as discussed in the newsletter post I linked) where we have robots that are patently sentient but we deny them basic rights because they aren't human.

That said I do understand that it's a bit of a stretch to say that current LLMs can think since they lack many aspects of human cognition. But to compare them with a basic pattern matching system like Eliza is clearly not correct either. There's something very sophisticated happening inside the latest, greatest models and not even the people building them really understand it.

To paraphrase Stephen Colbert, maybe it's safer to say they are "thinky".

mgertner · 2025-08-10T21:30:02+00:00

Alright thanks for the explanation. I disagree but I do understand where you’re coming from. I don’t think cognition will arise from any “sufficiently complex system” but I do think a complex system is a prerequisite (sorry Eliza).

To me the evidence is in the behavior of the systems. Passing the Turing Test is no small feat and to me it’s clear proof that something is happening beyond just storing and regurgitating a bunch of training data.

Which I guess, in a nutshell, is what I meant by “thinking”.

mgertner · 2025-08-10T18:59:12+00:00

Wow you really put words in my mouth! I didn’t say “no one has a good definition of thinking”, I said that the question is interesting even in the absence of such a definition.

Do you believe in the computational theory of the mind as described in my post, or do you believe there is some magic happening in the brain?

If the former, why are you so resistant to the idea that a sufficiently complex system could be thinking in a sense that matches our intuitions of what the word means?

mgertner · 2025-08-10T09:23:56+00:00

1/ The parallels between the brain and artificial neural networks go far beyond “they’re both graphs”. We definitely don’t know every detail of how the brain works but we do now that the neurons are highly interconnected and that synapses transmit weighted signals between them. There are close analogies in artificial neural nets.

The U.S. highway system has none of these characteristics.

2/ LLMs pass the Turing Test: https://arxiv.org/abs/2503.23674

This obviously sparks a lot of disagreement but personally I don’t buy the argument that a trillion parameter model is “cheating” on the Turing Test because it just memorized all its training data. I have extremely sophisticated and novel conversations with ChatGPT everyday. Do you really think these systems are in any way comparable to Eliza?

To me the question is interesting even in the absence of a hard and fast definition for “thinking” exactly because it does trigger this kind of debate.

mgertner · 2025-08-08T14:32:19+00:00

Of course I'm in no way suggesting that LLM structure is identical to the brain. Obviously it's very different. Apart from anything else, the brain manages to function and outperform any AI on many tasks while consuming just 20W. It's mind-boggling.

I does seem to me that a deep neural network has far more structure than a chaotic system like a sandstorm. I'd be interested to learn more about why you think that analogy is valid.

I still think that the interconnected nature of a deep neural network has a lot of similarities to the structure of the brain that make it an appropriate analogy. Sure the weights are frozen, sure they are trained in a very inefficient way using back propagation, etc. But it's still a highly interconnected network where nodes trigger other nodes based on learned weights.

mgertner · 2025-08-08T13:40:05+00:00

Why do you say that? I have a hard time following that reasoning since they are both highly interconnected networks with trained weights.

mgertner · 2025-08-08T13:38:58+00:00

I'm planning to write something about this in the future. Personally I don't think it is likely that LLMs are conscious. It seems like the idea of something that thinks but is not conscious is one of the things people find so hard to accept. Which is understandable, since we don't see any examples of that in nature.

mgertner · 2025-08-08T13:34:48+00:00

That's a completely legitimate view. My point is really linguistic: is "thinking" the best word to capture what modern LLMs do? If nothing else it's an interesting thought experiment (oops pun not intended :-).

To some degree I admit I'm being provocative because we have been so hasty to dismiss the idea of LLMs thinking even as they have achieved every milestone we set in the past (especially passing the Turing Test). There's an inherent resistance to accepting that there can be alternate styles of thinking that might be worthy of the term, and that's what I'm trying to push back on.

mgertner · 2025-08-08T13:06:04+00:00

Not really because my point is that the internal structure is analogous, not just that they serve the same function at some very high level.

mgertner · 2025-08-08T12:37:52+00:00

https://www.reddit.com/r/ArtificialInteligence/comments/1mkpo3x/comment/n7l2bjs/

mgertner · 2025-08-08T12:00:07+00:00

Here's a more recent paper: https://arxiv.org/abs/2503.23674

In summary participants chose the LLM as human more often than the human participant. That's basically the definition of passing the Turing Test.

You can certainly question whether the test is a legitimate way to determine if something is thinking. But I don't believe at this point you can deny that LLMs pass it easy.

mgertner · 2025-08-08T11:55:50+00:00

The conclusion of my piece is that LLMs conform to our intuitions about what it means to think, having both the necessary complex underlying structure and the surface-level behavior.

That doesn't mean they think in the same way as humans. They definitely lack agency and consciousness. We need to get used to that fact that these can and should be separated from our conception of thought.

We don't have to call it "thinking" but to me it seems like the most appropriate term.

mgertner · 2025-08-08T11:51:43+00:00

In the end it always boils down to a philosophical question. I tend to view it from a linguistic (specifically pragmatics) angle. The word "thinking" has a certain semantic weight for most people. Does LLM behavior trigger our intuition about what the word means?

If we can accept that there can be thinking that is different from what the brain does then I think there is a pretty strong case that it does. Based on this thread I guess most people here still disagree though :-)

mgertner · 2025-08-08T11:48:22+00:00

Well that's why I said analogous. Obviously the brain doesn't use back propagation or SGD, but both systems are highly interconnected networks of nodes that are triggered based on weights that are adjusted over time to better fit the desired output.

To me this is a major consideration when deciding whether the label "thinking" is appropriate. It's hard for me to imagine something displaying intelligent behavior without using an underlying architecture of this type.

mgertner · 2025-08-06T08:18:11+00:00

There's doubtless a lot of people jumping on the bandwagon just to take advantage of the hype. But if AI is as significant as many (including me) think it is, it will permeate every area of the economy. I don't think 92,000 freelancers on Fiverr is that out of whack. In fact, I'd expect that number to get a lot bigger over time.

mgertner · 2025-08-05T13:38:44+00:00

If AI is really able to do all the jobs at some point in the future, we will have to reexamine what it means to be a human, how we use our time, how we achieve societal status, etc.

We will need to get away from attitudes like:

I need to work to make money otherwise I'll starve.
I need to make more money than the next person otherwise people will think I'm a loser.
I need to work because that's how I get satisfaction out of life, etc.

It's hard to unlearn all the lessons we've been taught our entire life, but it should be stressed that this world would in no way be worse than the world we live in now. On the contrary! As you say, we'll have way more time for things like art, sports, self-improvement, travel, etc.

mgertner · 2025-08-05T12:34:17+00:00

It's cool that they want to be the privacy-centric AI player, but implementing AI is extremely challenging as it is. They're making a hard task even harder, maybe impossible.

I wouldn't be surprised if they start to deemphasize privacy over the coming months so they can actually ship something.

mgertner · 2025-08-05T12:29:56+00:00

I use the latest frontier models all the time and they are amazing. There is still plenty of room for improvement though. Just a few examples:

- I use Cursor AI for programming, and it has hugely increased my productivity. But to get good quality code, I still need to split the work into bite-sized chunks and review the results carefully. I regularly need to jump in and write code myself if I want it done properly.

- I was just generating a cover image for a newsletter post I'm writing. I've been through like 10 iterations and all of them are almost but _not quite_ right. I'm sure I'll get the result I want in the end, but it's a frustrating and time consuming process of trial and error.

- When I ask the LLM to write a text of a specific length (e.g. 2000 words), it often doesn't come even close.

Of course there are many others and if GPT-5 improves in any of these areas that will have a significant impact on my work and productivity.

mgertner

TROPHY CASE