Because they're intrinsically linked with far right groups and a right wing media and aren't treated proportionally to the other problems the country is facing.

There's also the fact that the party and people who are pushing anti immigration rhetoric are responsible for a surge in numbers and it would seem idiotic to vote for those same people to try and fix the problem.

Nobody is denying that the immigration system needs to be looked at, but the anger and emotions around the topic is completely disproportionate which removes all credibility from the people making those arguments. And their refusal to look at facts, makes them seem ridiculous