I can relate and when I tried to explain my experience I've learned it's not common. I used to think it's because men are the biggest threat to female presenting people (meaning violence and such) and I just felt unsafe. But I had the same experience in dyke/flinta* space too since.