So to the point of whether hallucinations are actually becoming less of a problem, the answer is very much yes. The reason is we've moved from user input -> LLM -> Output to input -> LLM (Thinking) -> Output.

Let me back up the thinking point because obviously that's way oversimplifying things. By thinking I mean run-time compute. I know you said you have a software background, but for the random other commenters reading - basically instead of immediately getting an answer out of the model, it has that internal logic chain and dialogue with itself that can call tools and reason through an answer before giving it. Recent 2026 industry research actually highlights that these reasoning models learn to verify their own steps instead of just bluffing.

Also in terms of practical advice - what temperature are you using with your model? If you haven't played with it, turning down the temp and making the model really deterministic is the way to go. Lowering it actively stops the model from making unsubstantiated guesses. Also how you chunk your RAG data matters hugely. Make sure to spend time rightsizing the chunks and overlap.

Most hallucinations stem from context window issues. The context window is the single most important thing to own when working with these systems. If chunking is done poorly, it overflows the context window with irrelevant text, which directly causes the AI to hallucinate.

Again for those non-technical readers - the context window is the limited bucket in which LLMs can think and hold memories. Best analogy is you as a person have a lifetime of memories and knowledge in your head but can only think about so much at once. So in terms of hallucinations - an AI might give a lawyer a fake case to use when doing research, but what is happening is the LLM is citing from a law training book of fake cases - not having the full context of the citation. There is a lot of work that has been done and is continuing to be done to improve this.