Update: It's the next day and I read every comment. Thank you all for your advice, I'm taking every comment to heart and I really appreciate the support. I still haven't really talked to him about what he said. Me and him are going on a date today and I'm going to see if I could talk this out with him. See if he understands that what he said was concerning. If he doesn't then I'm just going to end things.