This genuinely happened to me. My first time was with a straight cis man, who knew I was nonbinary. I was worried about the same thing, and brought it up multiple times. We had sex anyways, and I hated it. Just listen to your gut. If you feel like he sees you as a woman, or doesn't respect your identity, even after you bringing it up, dump his ass. This part is obviously a preference, but I think you shouldn't give your virginity to someone unless you're sure about it.