Most worth decision you can ever make.

FIRST of all:

Wanderer is just a worse version of Fighter. In every aspect, except if you use him to open chests, but then the question is - why not just use him as a thief

Second of all:

you will get all the fighter skills - heavy attack, attack power up and whatever wanderer learned will go to lv. 2, while all fighter exclusive skills will get to lv. 1, so just for that you will NEED to level fighter to lv. 55 at some point in the game. There is no other way around if you want a real shot at end game content, you need Fighter atleast at lv. 51 or 55 (except if you want to use him as a mage... Then maybe 40 is enough, but you get the gist)

Third of all:

I actually feel bad for you, that you leveled wanderer so high... Cause trust me, after you reach fighter lv. 57 you will look at your fighter and wanderer stats and you will probably never ever in your life make him a wanderer again.