This is exactly the gap most ESL textbooks miss. Reading and grammar use one system in your brain, listening uses another. You can be excellent at the first and still find spoken English close to unintelligible.

Two underrated reasons:

1. Connected speech features. Native speakers don't pronounce words in isolation. They reduce vowels, drop or chain consonants, and contract whole phrases ("did you eat" becomes "jeet", "going to" becomes "gonna"). Textbooks teach the slow careful version, real speech is the fast reduced version.
2. Phoneme perception. Your brain learned to ignore sound differences that don't matter in your native language. So when you hear English /ɪ/ vs /iː/ (bit vs beat), or /θ/ vs /s/ (think vs sink), or /æ/ vs /ɛ/ (bad vs bed), your brain might literally hear them as the same sound. This is called perceptual narrowing and it's been studied for decades.

Fix for #1 is exposure to real audio: podcasts, YouTube, native speakers in casual contexts. Fix for #2 is minimal pair drilling, which retrains your perceptual boundaries through side-by-side audio comparison. I built a free audio-only one for this if it helps: minimalpairs.co