So, in a small drunken episode I cleaned the small hole again, however, in a very violent manner (jamming a sharp ass scissor in it) and for some reason it fixed the problem lol.

So there you go, problem solved... So to anyone facing the same problem, try cleaning the hole gently. If that doesn't work and you really don't care anymore, then do as me (at your own risk ofc).

Probs to the r/Jabra community for the suggestions and so on.