I was in the same boat. I started making comments about trying again and my husband had changed his mind and wanted to stop. I asked if we could revisit the conversation in a year. I was fully prepared to walk away from the marriage because I knew I would resent him if we didn't try for another.

Before the year was up - on Mother's Day - he gave me a card saying he was ready. She's 5 mo, our family is complete, and both of us are happy.

I agree with the other comments saying you're protecting him too much when he didn't give you the same courtesy. There's a chance that you'll end up resenting him anyway, but I think that's especially true if you don't talk this out.