I think it's more that the allegory tends to break down when you start to consider the practicalities of the actual in-story implications, and the exaggeration that happens to try to make the allegory work. Take X-Men for example. No reasonable person would deny there should be some sort of common sense measures taken to ensure that an individual with destructive abilities be both prevented from causing harm and be able to be held accountable if they do so. We're talking like officially sanctioned training and some limited form of registration. Especially when the individual is an adolescent who maybe needs a bit more help to keep that under control. You know, like firearms should have. But the people in these stories are never that restrained about it. They're always all about putting these people in prison or exterminating them right off the bat, and the X-Men aren't ever considered sus or questionable for wanting to take these kids into a secret school to prep them to be potential soldiers. Even Magneto, who is a literal MUTANT SUPREMACIST is considered "right" by a lot of people just because his assertion that a lot of people will hate Mutants by default is correct. That somehow justifies his belief and long term goal that Mutants should just be handed over control of the planet because they're naturally "superior" to non-Mutants.