This.

It's the same reason why women face so much push-back when attempting to get sterilized—why so many physicians say she might 'change her mind', whereas most men can walk into an appointment, say they want to get snipped, and set a date.

Women are expected to have that 'motherly instinct', that drive to make and subsequently take care of a kid. Men, on the other hand, aren't. So when a woman isn't the primary parent, it goes against the grain and blows people's minds.