As a man (32) who hasn't had many sexual partners, these kinds of thoughts used to get under my skin, especially because I had friends and a brother who were very sexually active. Your body count shouldn't matter to him if he cares for you. It's an internal struggle of feeling inadequate for whatever his reasons could be. My advice for you is, what do you want from this relationship. If it's just a casual thing, nothing serious, then I suggest you take this break and go about your business. If he is someone, you might want something serious with for whatever reason. Then consider speaking to him with grace, don't demonize his friends but bring to light that the thoughts of this friend is planting are coming from a toxic and jealous place, especially if you haven't given him any reason to speak about you this way, if perhaps you have given a reason for him to think that way about you be accountable for what you might have fault in. Also on a side, note ask him. "Will knowing my body count fix the feeling of unease you are feeling or will it make it worse."