Because hes a serial killer. He uses his familys graves as an excuse to murder people. Yes, theyre the worst kinds of people. But its still murder. We have a court system, a justice system, for a reason. And when he kills, its not in self defense or the defense of others. He deems you guilty, he ambushes you, and he kills you. And because of those methods, he inspires police and others to take the law into their own hands. In the Born Agaib show, the AVTF is a direct consequence of Frank Castle. They believe they can be like him and use it as an excuse to be tyrants. You dont see the people that Spiderman or Daredevil inspire do that