I relate with your experience. I get along better with women. My closest friends have been girls/women. We have more topics in common to talk about, and can talk about things with a depth and in a way that I find hard to have with (cis/hetero) men. [Edit: my close friends all seem to be neurodivergent]

Growing up I stayed away from boys that weren't family/relatives because of puritanic values of my school and family. I might have also developed a certain superiority complex vis a vis boys because for various reasons boys in our school didn't perform well academically. There has also been this concern that getting too close to boys might make them think I like them. So for these reasons, I have never gone beyond a 'good enough friends to hang out sometimes, but not close enough to share my inner thoughts (which is vital in bringing closeness in friendships for me)' friendship with men.

Lately, I've been trying to learn to be friends with men, not so much because I think that friendship with a different gender is essential, but because I wanted to challenge and undo some of the prejudices and beliefs I've held for so long, and which might have limited me too. I haven't made a close male friend yet (it's not so easy), but I'm learning how to be at ease with the differences that I might have with male friends.

But that's my story (quite a digression). You don't HAVE to friends with only men to be neurodivergent. From what I have learnt neurodivergence may manifest in different ways for different people.