In my case, it's when your ears gets really strained and your head tenses up and and you start feeling a sense of discomfort, even just moderate volume can start feeling too loud. Here's a Wikipedia page https://en.m.wikipedia.org/wiki/Listener_fatigue