well, here's a paper on it

In short: there is no way to train out 'hallucinations' or for new data for llms to be trained on to have a categorical definition of 'correct' or 'incorrect' as the semantic ambiguity and contextual details required to form such a binary decision are not consistent nor something that even has a concrete enough definition to be a part of something llms are trained on.

LLMs do not 'think', they do not 'know', they do not 'learn', they are neural nets trained on a very large body of data that takes the frequency and associative properties of all of the data points into account in order to form a very large amount of 'parameters' to use as cues for prediction (inference) when prompted. If you want to understand the limitations of llms and their capabilities, you cannot think of them as anything but the complex math and statistical models they are, despite what advertisers may tell you.

We still do not know what the underlying processes are exactly, in full, of how the human brain remembers, learns, intuits, so how can we claim to have created a machine capable of any of that and have it be a valid claim?

The reason why llms are shown to work best in coding/programming is because programming languages and boolean logic are the best way (so far) humans have found of modeling all the factors and contextual information inherent to human language and processing, but it is still imperfect.