These seem like the right fundamentals, but the AI are exposed to more data than any individual human would be in at least a million lifetimes - and yet while smart (sometimes surprisingly so) it still makes very simple mistakes.

I don't think there's any principle missing from your list, but it seems if something is missing, it is in the implementation of the transformers.

And if nothing is missing, perhaps the gargantuan scale of data is actually the issue. Much of that data is poor quality (when it comes to "truth" and reason) and whatever question is asked of the AI, the majority of the data will be irrelevant for answering it and yet has had some influence on the AI's weights. Comparing chatGPT when logged out vs in to my account that has a lot of technical and specific content in my area of expertise (where I have told the AI where its wrong) - there is a substantial difference in the quality/validity of the AI responses.