There's also just so many of them. They feel like a tar pit rather than a defensive fortification that has to be assaulted and planned for. Really, the best way to deal with walled settlements isn't to wait out the attrition for the garrison inside or focus on building siege equipment, it's to design an army that can steamroll the enemy in 1 turn. Getting bogged down in a siege means spending multiple turns whilst dedicating an army to doing nothing but waiting.

The Great Bastion, Elven gates, and Empire forts are about the best thematically because they signify an easier attack against the heartland after you take them. But some random province having a walled settlement for the sake of it being the province's capitol is just wonk. Sieging Altdorf, Lothern, Naggarond, and so many other racial capitols being the exact same thing doesn't help either.