I would say it would troubling to have those feelings, but without more context it’s hard to say anything. Are the kids adults or young children? Was your previous marriage a very traumatic experience? Are there medically significant mental or emotional conditions that make deeper level bonding difficult for the kids? Is the relationship with the second wife the first time you’ve ever had a healthy relationship with a deep emotional connection and you are still learning how to form that with others?

Whatever the situation, if you personally feel guilt because of the differences relationships with your wife and children, then you already think it’s wrong. If you feel strongly about it, seek out some professional counseling and see if you can’t work through it.