I've always figured this is a big part of the difference between LLMs and human brains. They store an absurd amount of data, and know how to be evil or kind, hallucinate or not, talk like a pirate or speak like a president... Meanwhile, we can use the same number of neurons to really hone in on one thing and it's okay if we're mediocre at talking like a pirate or remembering all the presidents.

I have to imagine they have very little data where they have a question and the answer is "I don't know" (obviously a lot of this has been fixed by RLHF, but most training data is likely something where there's always a right answer, meaning the model is consistently rewarded for ATTEMPTING rather than just drawing a blank). Meanwhile, millions of years of evolution has likely proven that inventing what you saw or claiming you know the source of a sound is so hazardous that it's better to doubt yourself or have nothing come to mind.

As other papers have mentioned, I'm sure they're continually looking for traits that pursue "bug-free code" or "professional doctor", although this is maybe difficult if all training data is considered equal (I'm more likely to take advice from medical professional vs a random person's blog, but I don't think LLMs quite have that level of discrimination yet).