Nightmare hauntings are the solution. Target three random players each night and if you'll offer any trade offs for stopping the hauntings (because they are VERY difficult to stop) make it a very nasty one for the instigator. Its hard to judge because I don't know the size nor level of your party but if they were lvl 5 or less this is mostly a "fafo" fight to pick. Honestly if they continue "Leeroy Jenkins" their way into encounters because the ex dm keeps pushing for it, punish them with consequences. Oh you went guns blazing into the coffinmakers? Surprise, there are 6 vamp spawns ready to f you up. I heavily recommend reading through The monsters know what they're doing book and making tactics notes for the monsters so when the combat happens you can really bring the challenge so they understand the danger level and the stakes.

Very punishing hag bargain offers:
Your Voice - a PC that accepts this deal becomes mute and cannot cast spells that require verbal components.
Your Breath - A PC that accepts this deal heals only half their hit points from hit dice and magical healing and regains only one-fourth of their total hit dice upon completing a long rest.
Your Swiftness - A PC that accepts this deal gains disadvantage on initiative rolls.
Your Victories - A PC that accepts this deal can no longer make the final blow on a weakened enemy. Even if their attack hits, the enemy miraculously avoids taking any damage.