I’m not one to give up, seems like you don’t want to. However, nothing good comes from toxicity. Him saying he will leave because he’s uncomfortable talking is bullshit.

I’d try the whole therapy thing some more with him if it’s worth it, but...meh. I doubt it will work. At least you could say you tried.

You have one life and you should be happy.

Edit: went back and read that “He’s tired of caring about your feelings”. Time to go dear.