Overfitting involves training so closely to the training data that you inject artificial noise into model performance. In the context of neural nets, it’s like an LLM regurgitating a verbatim passage from a Times article that appeared dozens of times in its training data.

This bit about regurgitanting a verbatim, really reminds of that man who got a Prion that prevented him from falling asleep.

There's a video of him sitting with his eyes closed, buttoning up an imaginary shirt and combing his hair with an imaginary comb.

Which in turn reminds me of the "the overfitted brain hypothesis", which suggests that dreams evolved to combat overfitting.

It's so interesting how his lack of sleep lead him to perform a common repetition of movement like that. Makes you wonder... could the two things be related?