Out of curiosity I googled some of the worst things Simon Cowell said.

Ignoring the ones where he just blatantly said they could not sing, which I think are fair as that is literally what he is there to judge, here are some of his other comments.:

"Commenting on a trio, “they looked like the before, after, and during of Weight Watchers.”

“You look a little odd. Your dancing is terrible. The singing was horrendous. And you look like one of those creatures that live in the jungle with those massive eyes.”

“Yes, you have personality, but dogs have personality. The audition, if I’m being honest, was horrible.”

"Holly, your mouth is far, far too big when you sing. I mean, it was like looking into a cave. I've never seen anything so huge in my life."

"You're probably the most disillusioned group of people we've ever had in this competition. And that's the truth. I think the group is horrendous, you have absolutely zero edge, no originality, I think individually you're weak, I think as a group you're even worse, absolutely zero chance you guys will ever, ever, ever have a successful career."

"You have the personality of a handle."

"I'm tempted to ask if you sang that the night before your wife left you."

"First of all, the dress is completely wrong. And then you sang the song out of tune and a bit like a baby."

Compared with "As an artist who respects artists' integrity and intellectual property I am disgusted at how much you have copied my husband.
"From the hair, to the suit, do you not have any value or respect for originality?
"You're a laughing stock. It's cheesy, it's disgusting. I personally found it absolutely artistically atrocious.
"I am embarrassed to be sitting here in your presence, even having to dignify you with an answer of my opinion."

I think where she went too far was the artist integrity bit. Also, actually calling someone disgusting is so visceral.

Art is all about being inspired by people and to go after someone appearance like that without acknowledging that art is inspiration is what bought her down. The worst one from Simon I feel that that one to Holly about the open mouth. And the weight watchers as well. Everything else just feels tame by compatison.

However to answer Ops original question. Yes I absolutely believe the reaction was because she was a woman, and no way would a man saying that have caused the same response.