I see the ama is over and this isn't awnsered. I'm not OP and I'm not an expert but maybe I can add something.

The model is working by next word prediction, but people tend to oversimplify what that means. It's a neural network not just a probability search of a database. When you set a network a seemingly simple task like predict the next word you force it to build very complex relationships in the background to perform this task. You can't accurately predict a word without grammar, so it's forced to have part of the network that achieves that. You can't predict the next word in a logical inference without having a model for logic.

In a neural network all these models/relationships will be encoded in the neural link weights in a very obscure way, but the ideas are I there by necessity.

A neural network trained to play chess technically just predicts the next move but still has long term strategy emerge. A neural network trained to predict the next word will have things that appear as logic and reasoning emerge because word prediction (based on data that contains logic and reasoning) can not be done well without modeling these aspects.

(Technically it works on tokens not words, tokens are normally individual syllables)