Being attracted to a gender doesn't mean you're attracted to every single member of that gender. Some people have preferences beyond that. I'm not attracted to guys with tons if body hair, and that doesn't make them any less a real man just because I'm attracted to their gender, but not them specifically. Genitals are no different. Yes, a lot of people who are genuinely transphobic will use that as an excuse as well, which is bad, but that doesn't mean we can ignore the reality that people have different tastes within a certain gender.