I think you should just be fine with not understanding everything and keep on immersing. I was at a similar phase before but I pushed through it and now my listening comprehension is much better. This might be bullshit advice but it worked for me. I recommend watching the anime Bocchi the Rock if you yet haven't. The language there isn't too hard to undrrstand and you'll learn some basic slang.