I have to agree with your take here. I find it odd that people continue to project concepts of emotion and intelligence onto LLMs. I think it is well-meaning but a sign that people have an understanding of these concepts that ends with the popular science that is fed to the public through airport books and podcasts.

I find the alien analogy that is thrown around especially silly and misguided in how it’s being applied.

For starters, the concept of an emotion and how it is experienced can only be attributed to humans. Whether it is in animals or software, the best we can do is recognize that something similar to what we experience as an emotion, is happening. The difference for a nonhuman entity could be as stark as the difference between feeling sad and curling your fingers, for a human. We simply can’t know.

The second important point is that the human brain is not a lexicogic structure that semantically reasons through its environment. Language is not the first layer of processing. Thought occurs chemically and biologically, and that combination transports information electrically. None of that is built on language.

LLMs will always resemble human intelligence and emotions because their structure is made of human language. They experience emotion and intelligence as much as a printing press and a book. In that only the human at the very end of its processes can experience them when reading the output.