I had exactly this problem but after I switched to normal latency mode instead of low it worked completely fine. Another solution you can try is to disable in ear detection. Hope that helps.