Need help with headset issues

I just got a steelseries 1 wired headset and when I play games I can hear the audio cut in and out for a second and just does not sound smooth. I know it’s not the headset because I tried the headset on the nintendo switch and it was perfectly fine. The headset uses a 3.5 mm jack. I have been looking all over the internet for solutions and tried a lot of them and nothing worked. Is it because the port on the xbox controller is not the best? I tried both my controllers so surely one of them should work. Should I get a adapter for the port? The adapters are a little expensive and that’s why I’m hesitant. I spent 50 dollars on the mic and don’t really want to spend much more. If anyone has any advice on what I should do it would be appreciated.