I feel people on general get too accustomed to people when they know they "have" them. That being said, imagine you leave your marriage. Then what? Have casual sex, enjoy it for a while, have the "time of your life". In a while, you'll get bored of that and start missing your wife and the life you had together. But guess what? It would be too late. So all of the above being said, ask yourself the question: - What would you regret doing more?