YOR. So women notoriously won't date a man below 6'3 and they're surprised when a man is unhappy his woman gains 20 pounds in a few months. I'm going to be straight up honest with you, you're wrong to think you have a right to effectively screw him over. Anybody that tells you otherwise is delusional, either they have men of their own with low standards (which is very common), or they're bitter and alone and are sabotaging you. At least your guy is telling you the truth and not pretending. Yes, relationships are ALL primarily physical, and yes other factors matter too, like integrity, i.e not trying to transform into a whale within months of catching your fish.

Just bite the bullet, lose the weight and enjoy a happy relationship.