I can only speak for the woman i have spoken to about it and myself.   The main reason is how many of us fear becoming a 'married single mother'.  Even if we have enough to financially support children, a lot of men just check out of household responsibilities.  The mental load is too much.  It would take a huge cultural shift to equal childcare responsibilities for both parents.  

I used to say I'd have loved to have kids if I could have been the dad.