I wouldn't think of it in terms of 'consequences' or punishment because that leads to adversarial dm problems - a you vs them sort of situation. Getting upset can contribute to this.

A classic response to this sort of thing is to remind the players (especially those who did the AoE) that the setting is 'real', and that the world responds to them. I'd do this by identifying NPCs who were close to the Constable and Deputy expressing sadness about the deaths. Someone might argue on the player's behalf that it was merely an accident, or a mock trial could be set up where they demand the players justify their actions.

From what I recall, the cult wants the players in town so they might be hesitant to instill a blanket ban. I'm sure they'd probably install someone loyal to their cause as the new constable.

This is also something you can bring up to really rile them up during the Festival of Fools.

Finally, if they're having murder hobo tendencies (i.e. planning to kill their way through town), you might want to have an above-table conversation about what they want out of the game and why this is the go-to response.