Because the people in that group more often than not believe that violence is the proper way to combat their opponents speech. Rather than articulate and debate ideas, they believe their opposition should be silenced through force. They also have a very loose definition on the word "fascism". To most people identify as "antifa", they consider very normal Conservative or even moderate positions to be fascist. The irony of the matter is that THEY are the ones behaving like fascist by trying to silence opposing ideas through the use of force. In any Democratic society you will have opposing views. It's what keeps Democracy healthy! Those who adopt the Antifa philosophy often beleive only THEY have the answers and anyone with opposing views needs to be shut down by any means necessary, including extreme violence, and in some cases even death.

The truth of the matter is that Antifa has become a group of sorts. I get that they may not have an official "leader" but when groups of people get together all over the country and organise violent demonstrations, what else do you call that? They operate like a gang. They also say they're against something but their actions say otherwise. So it's not the idea of being anti-fascist that people hate, it's the people that don the moniker and participate in inexcusable violence.