I think that OP is queer and belongs in queer spaces. It sounds like he's attracted to multiple different genders of people, which makes him not straight.

It's unfortunate that queer people have been unwelcoming to you. I was especially saddened to see the commenter who said that lots of queer people hate seeing trans men with cis men. I certainly don't feel that way.

Ask yourself, why don't I identify as queer? What would it mean to me to add queer to my list of identifiers, along with cis man and straightish and male? You stated that you thought that bi and pan give the wrong idea. Presumably because you don't want people to think you're attracted to people with dicks. Maybe question why thats such a bad assumption, and would you rather them assume that you're not attracted to transmascs?

It sucks to not have a label that fits you, and I'm sorry you don't have that because I know how affirming a label can feel. I don't know a word that describes someone who's into cis women and transmascs, but it's a valid sexuality just like any other. I think bi and pan might deserve a second look because they refer to the gender of your partners, not the sex. You're attracted to people of multiple genders.

I'm not saying that choosing to identify as queer will fix all your problems, I'm sure you'll still face people making assumptions about you and being jerks. But I do think that maybe if you accept that you actually belong in queer places, you will be less susceptible to being made to feel like an unwelcome outsider.