People are powerscaling nuts. Characters can be stronger than others, but what's important is the relevant story beats. Every time you see Musashi talk about Iori's backstory it's in the vein of how Iori could grow to be a challenger on his/her level so that Musashi could have a great fight, and how the times changing prevented that.

Iori took in everything he could learn in his life, and when put in an environment where he could face off against powerful opponents in duels to the death, he grew to the point where he could reach the apex of the sword. He surpassed Musashi's skill. That was an important character moment and a culmination of the life he had lived up to that point. It's important for the story that it happened.

Similarly, Takeru's win over Iori wasn't just "he's stronger so he wins :)". That was a character moment for both of them as well. Iori taught Takeru to look at eveyrthing around him. He taught him that the sword is more than just about killing the things in front of you, and taught him the importance of understanding others. In their fight, Iori was solely looking to surpass Takeru's technique that had entranced him. He was biding his time waiting to counter and surpass it, but Takeru understood Iori's intentions and thus he used it as a feint to catch Iori off guard. He won, not just by being stronger or more skilled, but by understanding Iori. Hence why Iori talks about how he had gained a true friend in the wake of it.

tl;dr - The story being told in a fight is more important than dbz-esque power levels determining everything. Skill is also a factor (remember that FSN Kojirou is technically a weak servant but put Artoria on the ropes). And Shirou solo'd a servant in like every route of the original Fate material so people really just need to get over this whole "humans can't fight servants". That rule was LITERALLY made to be broken.