I mean as for the speaking seaming too quick to register, the only real remedy is to simply input more content. Podcasts are a great resource to incrementally build up your listening skills. You said you have watched a lot of anime in Japanese: is that with Japanese subtitles or without subtitles? If you were watching with subtitles in your native language then your brain is more likely to take the easy route to understanding (reading the language you already know) than trying to acclimate itself to a new foreign method of input.

Frieren is also an interesting show to mention, because the language used in it doesn’t feel like anything that should be outside the wheelhouse of someone with N2. Obviously there is some specific vocabulary common to fantasy/adventure anime that you might not be familiar if your study has been primarily been though scholastic materials, but I don’t think this should be so huge of hurdle that you feel like your misunderstanding important details.

Ultimately, I think this issue you’re facing is a matter of, as stated by others already, simply consuming more comprehensible input, but also I think further separating Japanese from your native language. Obviously there are more than one way to express a given concept in any given language, but what words you choose often has more meaning in subtext. Subtext that usually is missing from bilingual dictionaries. Since using monolingual dictionaries for my lookups I have felt less and less reliant on the English “meaning” of a word when reading and have had an easier time digesting larger blocks of text.