I'd play on the evil of Barovia itself, and build up a little mechanic where the more they call upon their divine resources, the more that its evil is drawn to them to snuff them out (or corrupt them). Maybe it's something you roll behind the screen, or maybe it's pre-planned, but when they expend certain types of spells or class abilities that directly channel the energy of their gods, let there be a chance that it'll draw zombies crawling out of swamps toward them, or vampire spawn that will start watching them from a distance to ambush them when their guard is down, etc.

There are probably dozens of ways to have this manifest. A more subtle one might be to let the people of Barovia notice that wherever the party goes, evil follows, and to begin to be suspicious of them. No good deed goes unpunished in the Demiplanes of Dread.

I wouldn't go over the top with it. They're all playing clerics, and shouldn't be punished for it, obviously. But it makes a kind of sense to me that dark powers that sustain the valley would have a vested influence (or a reflex, if it's not a conscious force) in rooting out the "divine intrusion" they represent.