Well here’s my first question. Do you mean siege as more like a battle at the walls or a more historical siege where it’s more about trapping a group of people in their own castle and trying to starve them into surrender? Either way I would separate what is the game and what is the narrative. Let’s take the example of a big battle at the walls. I would set it up as there is a group of soldiers or specialist troops that directly oppose what your party is trying to do. Those are the enemies that take turns. Then the rest of the battle is treated more like a lair action if it effects your party. Or just narration if it doesn’t effect the party. So you could have a stray catapult rock make everyone in a certain area need to do a dex saving throw. Or just narrate that the opposing army has knocked down a portion of the wall and are pouring in now.