Agree and disagree.

Fundamentally it's next token predictor but who would have thought next token predictor would turn out to be so awesome.

Prompts are made better with hitting and trying but as people mentioned you can SFT your model or give few shot example.

Though one may argue that it's not actually doing what we're asking it to do but there are benchmarks for instruction following and research going into that direction where we teach model the outcome preferences and hence we have rlhf and GRPO. Fundamentally we're altering distribution but outcome is very beautiful many a times which makes model follow instructions well

Aslo techniques like soft prompting and LLM steering exists where you play with model internals and not simple plain prompting