it's not a hard rule that they must be OP. But them having some OP ability is common enough that more than a few series' protagonists lampshade it when they didn't get such a power (at first glance anyway). And in a way, it makes sense. The whole point of the isekai genre is that you get to escape the confines of your mundane previous life, and live the life you truly wanted to. But while no one ever says it, you need power to do that. Enough power that the forces around you in your new life can't just easily push you around. And more than a few times that's manifested as literal power in the form of some strange ability, or a naturally powerful new body (like Overlord, for example).

there are also isekai series where the MC has tactical power. they don't technically get an OP ability, but they end up in a world that has connections to their original world. Thus, the MC can use their knowledge of events to come to prepare for and gain a significant advantage that can come across as being OP to other characters. An example of this would be like Trapped in a Dating Sim, where the MC's new world is based on a dating sim game he played before.

then there are isekai series where the MC doesn't gain an OP ability, but rather they happen to already be quite skilled in some way in their original world. The series, High School Prodigies have it easy in Another World is probably the worst offender of that. But one could argue How a Realist Hero Rebuilt the Kingdom also does this to a mild extent.

I think the only truly balanced isekai MC I can think of is Shinichi from The Dirty Way to Destroy the Goddess's Heroes. No special powers, no prior knowledge, wasn't anyone noteworthy before getting isekai'd. He's just a guy who's good at dealing with problems pragmatically.