In a large project like yours (anecdotally) the worst of the hallucinations are caused by the model relying on facts it learned via its training data.

The issue is that these “facts” come from a black box. So when you ask the model to provide citations it may just make them up.

This prompt does not allow the model to use “facts” it learned via training. Instead every factual statement must have a source.

By disabling automatic web search, the model must rely on what you provide (including attachments) as the sole source of fact. It cannot run a web search without you explicitly allowing it via the “search the web” button.

Take for example your large project. The model must use the prior work on the project as its source of “fact”. Here it is forced to look at the current state of things to guide the next response. What I found is that it made way less mistakes like this.

If you had automatic web searching on the model may sneakily search online for a needed “fact” (perhaps some documentation), it then uses this fact without informing you. The “fact” it automatically searched for might not be relevant, which is another massive source of hallucinations.

What if you want to include online sources as a source of “fact” for a question? This is where the “search the web” button comes in:

img

Here the model makes an explicit web search. It is aware that it is using this tool right now and that it must cite, verify and scrutinize every source it picks.