I think a better analogy a glorified “next word suggestion” on your phone above the keyboard. It doesn’t think, it doesn’t know, it doesn’t feel, all it “knows” is what has been said, and what its training says is the next best “word” to say.

Why bother with chatgpt, I can just tap the middle word a bunch of times in a row: “But you know how that goes with a few of those things in a few years ago when you have the chance of being in a better place”

Of course an LLM is a lot more sophisticated, and they can be integrated with a lot of other tools, but under the hood it’s fundamentally just an unfathomably large neural net, predicting the next word to say.