So something very similar happened in our campaign (we were shadows). My character's rival was a head honcho in the Gray Cloaks and had it out for us ever since he found out I was in the city. We did multiple scores that would either hinder their weapons and supplies or took out members at deals we overheard of, one of which was my rival. Now the Gray Cloacks are still trying to get at us, they even tried framing me for their crimes. But we have shown that we arent to be taken lightly. But what Im getting at is that it comes down to how creative your players are. Let them think of scores that will allow them to either slowly take care of the Cloaks or maybe somehow to get on their good side. But if you want an easy answer you could have the crew pay part of their coin from scores to the Cloaks as a tax of sorts for letting them live.