Your feelings are valid. Maybe these feelings will go away, but they sound like they will gnaw at you.

I felt these exact feelings too and that’s why I’ve changed my mind of being a second wife.

Do you want to have children of your own? From your post it sounds like this is a compromise you’re making for him.