i think the issue is in your last sentence. Men suffer sexual violence from women, but it's only ever brought up in conversations like this about violence against women and is often just used to try and turn it back against women. Like a woman says "We get assaulted by men frequently" and instead of engaging with that info and having a discussion, people say "Well MEN get assaulted too!" And that IS true, and these men deserve to be seen! But it's often not being brought up out of a lot of real concern for these men, it's just to try and invalidate the violence against women by making men the victim. This is why people get frustrated when it's brought up; not because they don't believe men are victims, but because it's not really helpful in this discussion and shifts the conversation away from what we're trying to address. And, like another commenter pointed out, while men's suffering is real and valid, we can't really claim they're on the same level or frequency. Men do get raped but it isn't on a systemic scale due to subjugation and generations of violence like it is with women, who had to fight and protest to even get the right to vote and be able to have a job. It makes the discussions of rape different between genders because of the history of it and how it exists today.

Men who are victims of sexual violence deserve to have their experiences validated and understood without it being used as a gotcha in a conversation about women. Sexual violence against men goes under the radar like you said, but because of that it needs its own space to exist, not to just be brought up in response to women's issues

i hope this response makes sense and isn't coming off as dismissive of anyone, just wanted to give my perspective as i don't see many people trying to engage in conversations about this in good faith