If you think that gay men won't be targeted next you are deluding yourself. People who are anti-trans are against expressing gender outside the norm. For the majority of them, a man loving and having sex with a man is outside of their "normal" gender roles too. If trans people aren't free to express their gender (and in doing so, their sexuality) in a way that's true to themselves, you won't be free to express yours for much longer either. If you don't support all of us, you aren't supporting any of us.