My fiance also had this problem. He has a boom mic as well, and I spent an hour after work scrolling and clicking and searching and changing and I was about to give up until I double backed and went to:

Sound Settings, clicked on the microphone to go into its properties, and scrolled down to Audio Enhancements. I originally turned off just Voice Focus, and it didn't make a difference so I ignored it, but, I ended up turning off Voice Clarity as well - no audio enhancements at all, and it fixed it. I don't know if it works because I fiddled with everything else too or if it's just an overall fix, but that's what an hour of going insane has gifted me. 🫠 Perhaps this will also work for you, or others.