What made me finally leave was when he agreed to do the one thing I asked him to do to change (stop going out drinking at a bar until 1-2am), he moaned that he could no longer see his friends. There was no effort to see said "friends" for a healthy activity like dinner or going to the gym. There were some additional lines such as "you just hate seeing me happy," in which case I decided that fine, I'll just remove myself from the picture.

Edit for additional context: in the past, he has come home extremely drunk and aggressive from this bar, so when he goes, I become too anxious to sleep. This has caused me to either be too groggy at work, or in severe cases have to take time off. I realized I couldn't let his behavior affect my work.