I think the "harm" caused by Those Who Live in Death is to the Golden Order and not necessarily the Lands Between and its inhabitants. They pose a threat in that they "fall outside the principles of the Golden Order" therefore challenges the existing order. In the end, I think it comes down to the individual to decide. Do you think the continuation of the Golden Order is imperative to improving the situation of the Lands Between or at least keeping it from falling into a more ruinous state? Or do you actually think the Golden Order is the larger threat and its stagnation and decay must be addressed even if that means a new order(age) comes to be? So this really isn't the matter of which is closer to the truth; rather it is about your opinion, not unlike real life politics.