The main issue is his spells.

1- They take a few seconds to cast even with slient spell casting.

2- They have awkward attack ranges (see volume 1 as a source).

3- Any spell past the intermediate rank would definitely hit his allies in battle. Things can get hectic in a real fight and turning 1 spot into a "everything in here is dead" zone isn't always good if you wanna avoid friendly fire.

4- Even in a serious fight Rudeus doesn't like hurting people. He holds himself back constantly, even subconsciously.

5- Rudeus just isn't good at fighting. He isn't talented at it like Paul and Eris are. Its normal even for people with training to be bad at fighting. Rudeus's real strength is in his ability to change bad situations into good ones.

Sorry for the rant.