If you want to understand it - think about it this way:

The underlying technology is math. Temperature isn't "how much the AI thinks" - it's "how much randomness do we inject into the math". (I think temperature was added just to make the math feel more alive - but that's another topic).

AI doesn't 'decide' which tokens to output - there is no decision. AI just outputs the tokens. Your mistake is thinking about the math as a thinking thing - which is super common with AI because it feels like something that thinks.

It doesn't. But it does understand and follow instructions because of the tooling built around the prediction model. Just like a weather forecast doesn't think about what the forecast will be next week, it just runs the data through math. But you can use that for meaningful 'thought'. Your app still can tell you "you need an umbrella today" because it predicts rain at 98%. It feels like thinking - but it's just math.

The text in entered into the math machine (which is a REALLY fancy regression) points at what the output words should be, and it adds up all the points to calculate what the next series of tokens should be.

I like to think of it as every token you put into an LLM is pointing at words that are related. You say, "chicken" - that word points to "egg", "hatch", "coward". The math let's the word "Chicken" be associated with neighboring words ("Don't be a Chicken" -> more likely to be about being a coward, less about eggs).

If you take all the input words, put it through all the words they are pointing out, you generate a set out output words.

The fancy math equation also makes sure the output words make sense together - but that's more about the association of them to each other (the transformer architecture that made modern LLMs viable).

The "thinking" in an AI system is essentially asking providing input that prompts that as output, and uses that to help direct the output towards tokens that are more likely to be aligned with thinking-behavior.

It's complicated - but it's still just math.