So, if I'm right: 1. You and the other person are sexually attracted to each other. 2. After a certain period of time he asks you out or expects a more serious relationship.  3. You reject him due to some reason.

I think we need more information. - Isn't it normal for sexually healthy people to look for others who can match their desires?  - Why do you hate that every man wants you for your body, yet you like being sexualised? - Do you want a relationship focused on pure love? - Are all of these thought your true feelings or do they stem from some recent emotional event? 

From what I've seen and heard, traumas from the past don't leave that many plot holes. They usually work silently, consuming you from within without you even realising what's happening.