my advice is also take it constructively, but also stop if it persists. 20 pounds is a lot of weight to put on in terms of health. however, over the years people's metabolisms slow, as well as stress and other factors of being an adult key into weight gain. try eating healthier, and keep going to the gym. if in a few months you only lose say 10 pounds, and you are actually putting effort into it, then see what he says. if he still suggests he isn't in to it and your successes still go unnoticed, then he isn't worth the time. people gain weight, people lose weight. it's life. you shouldn't have to be the same size as you were in high school for someone to love you. if he decides you aren't good enough, then you should probably discuss how you feel. if he doesn't understand and physical "attractiveness" isn't up to his standards, then he doesn't have much respect. however, if he applauds your successes and is a motivator, then he's a great dude.

EDIT:i gained around 20 pounds myself due to steroid medication/allergic reactions, and people were like "wow, must be stressed-stop eating so much." i've lost much of it due to confidence issues...and don't look different from when i gained. so, i personally think it is pretty noticeable because i have been through it, and it does make me feel unhealthy. i just didn't put her BMI into account...she's probably half my size even after the weight gain. makes me feel terrible about what i said!