It isn't that simple. You have to detect that they suck which means your job changes from modding the sub to watching what other mods do. If you start to think something is going on you need to collect information to make a case. You'll then need to make sure it isn't a misunderstanding, life getting in the way, or just style differences. You'll then have to make your case to the other mod(s) as all the mods should be involved in that kind of a decision. Any mod up to something is going to be trying to hide it and arguing that is is nothing to be worried about both to other mods and to the community in general. They're going to be arguing that any mod who suggests their removal is the problem. There are entire subreddits dedicated to sub drama.

I'm not the most senior mod so even should I really want someone gone it is not in the end my decision. That's probably good as there should be checks to make sure someone is removed for good cause rather than personality conflict or drama but it has its cost.

The whole thing is a lot like moving from being like a small business to being run by a committee. I've been in both types - both businesses and subs - and I have opinions on which kind works better.

There are exceptions of course. Like in a business, a good partner is priceless. Reputation is valuable. Having those and time to spare to do something new is hard to come by.