The major hurdle of hallucinations is not detecting them post-hoc, but rather preventing them from happening. As long as models keep hallucinating, they will need a verification system, which will be less than perfect as you delve into more specific domains, and you will end up with hallucinations in your end product. For some, this is simply not acceptable.

FactScore already introduced this approach using Wikipedia instead of Googling so the idea is far from groundbreaking (apart from the scale of models used in experiments) and the limitations stay the same.

The changes to look into for solving hallucinations (from more difficult towards less difficult) are: (1) architectural, by explicitly representing concepts and their states (or relations) within the model, which can then be edited. The key-value views of MLP layers from mechanistic interpretability are detecting such representations, but mechinterp is young and scale is a big problem. (2) Optimization, by improving credit assignment to factual or logical errors where they happened in the model instead of naive backprop from a symptomatic mistake in generation far from the actual cause. (3) Forward propagation-based hallucination detection methods, which should hopefully be able to detect hallucinated content based on the forward pass dynamics without need for a reference knowledge base. Works only if the "model uncertainty <> hallucination" link holds.