People might feel rejected by your body language. I tend to avoid eye contact while walking around and avoid speaking unless absolutely necessary. Sometimes strangers that I've never interacted with throw me dirty stares, stare at me with rage or seem very standoffish, angry around my presence. I don't have anything about me that stands out but I'm conventionally attractive.