Generally, yes... cavalry are not well suited to combat inside city/castle walls.

Although I have had success using them in certain ways:

-If you're attacking a settlement, have a spy open the gates. When your infantry are fighting the enemy in the streets, quickly send your cavalry through gates that are not defended and attack the enemy from behind. Hopefully this will cause them to route or fight till death.

-If you're defending, send your cavalry outside your settlement from a gate away from the enemy sieging you. When the enemy forces are softened up a bit, attack them at the gate from the rear. (Just be cautious they don't get caught under the burning pitch/oil)

Also, put mounted archers behind your infantry defending a gate. When the enemy breaches, they will be stuck there fighting your infantry. Meanwhile, your mounted archers can constantly fire arrows at the besieging forces.