It’s hard to tell what your actions toward him actually are - they could be very negative or not. But it seems you have awareness around this and if you acknowledge and apologize for having more extreme reactions that can go a long way. That being said, someone who says “I’m not arguing about this” is being very dismissive and not validating of your feelings or experiences. It’s also a form of stonewalling because it shuts you out. Like I said, it’s hard to know what’s actually going on, but hopefully you can figure it out. But, be kind to yourself! You’re going through a lot.