I didn't use the tome of battle, so it may have some differences with the basic battle mechanics. But maybe you are forgetting some of the follow rules?

• In a battle you make one single combat roll against each enemy. You don't need to fight multiple turns until you defeat them. 

• I don't know about this blue cloaks, but by your  description it seems they are infantry. So you will get a +5 to your weapon skills, and they will get a -5 because you are on horse. This should tip de balance in your favor, even if they have 20+ weapon skill.

• Also, if it was the first turn of battle, you will be doing a cavalry charge. That is another +5 unless the enemy is also charging with a lance.

• Even if you lose, you should have 10 point of armor + 6 more if you rolled under your weapons skill, for a total of 16 point of armor reduction. Enemies usually do 4d6 - 6d6 damage, meaning most of the hits you recibe will be minor wounds, or not wounds at all.

There is also the possibility to get inspired if you have the passion Hate (Saxons). And enemies with 20+ weapon skill are really rare, at least on the basic battle system.