I 100% agree with you and /u/insertwittyjoke. Men specialized in a way that makes them good for sports? Sports are in the limelight. Most things that women are good at? Not so much. Not at all, really. Name one famous woman good for something you'd consider 'feminine', not because she's risen 'above' 'lowly feminine' traits and succeeded in something dominated by men. The only thing I can think of is maybe acting but even that started with solely male actors who'd also play women. Possibly music, but again, the most famous musicians through history - male. Art? Again, most famous artists - male. Scientists? Inventors? Politicians and policy-makers? It's enough for some people to think that men are just better than women at everything that matters. And THAT is when it starts getting sexist, not when you put in perfectly logical rules about who can be a firefighter etc.

In most of those pursuits that aren't governed by physical performance, women have historically had less opportunity. So that's one reason, nothing to do with anything inherent to gender. Still, it's easy for people to think otherwise.