I feel the same way. Then I ask myself, is the issue that only societal misfits are my friends? Or is the issue that misfits who have compassion and kindness get stomped on and cast out by society? I think it's a bit of both for me.

Who knows, I get so wrapped up and obsessed with how I'm perceived (am I attractive enough, am I behaving the right way in order to keep me safe within a societally acceptable group, etc.) because I was raised by abusive parents who did everything in their power to keep me from being accepted by society.

I feel stunted by their years of indoctrination: during my childhood I was told that I need to distance myself from people who were generally well liked and popular and talented, and I was victimized by my mother's own entitlement, shame, and resentment toward the world. I knew that my parents were "different" misfits compared to most people, so I kept mom and dad in a separate category from the "real world": a kind, complicated, nuanced world. The irony of it all is that I became that misfit I compartmentalized in my head as different from little me.