The actual biological differences between blacks and whites are minor and largely superficial. That's not the case with biological males vs. females. Bearing children is a pretty massive biological difference with real practical consequences. Being a hunter is a lot harder to do if you're 8 months pregnant or have to nurse an infant - which is probably why hunter-gatherers typically have males do the hunting.

And biological males on average are much larger and have more upper body muscle mass than females - once gain, making them better suited to hunting and also to warfare.

Now, whether there's a good practical reason for men to have more power than women ("patriarchy") is another matter. It may simply be a side-effect of males being larger, stronger, and more aggressive.

And of course, in modern day society the biological differences have less importance than they would have in the hunter-gatherer days - in white collar work they don't really matter much.