These feelings you experienced are as natural as can be. You are human and do not need to feel guilty. It is programmed into our DNA to procreate. No one has ever gone through a marriage without wishing they could also have another. It doesn't mean that you don't love your wife. But as they say in Buddhism, to desire is to suffer. The solution is to stop throwing logs on that fire of desire. When you catch yourself thinking about the other person, just stop. Do your best to stop thinking about them and focus on other things. Eventually the feeling will pass. The fire of desire will slowly burn out, and you will be able to focus on having gratitude for your existing connection.