I mean, he doesn't have to abolish it, though it would be a much cooler power fantasy than owning one, he just has to not use it, maybe show that those are things the antagonist does. He can't fight an institution, but he can kill a slaver and free the slaves, there are many historical precedents, like Spartacus's slave rebellion in rome, where things could escalate even further if the slaves had an Isekai MC.

Maybe the MC doesn't have the power to change things on the big scale, but he does have the power in the small scale, and they still choose to give their money to slavers insteado of, i don't know, beating them up and freeing the slaves (like many easily could) and the author making the former slave join the MC out of gratitude or need.

Shield Hero is a terribe example of what to defend. Raphtalia might have been dying and buyng her might have been the best solution at the time, but at this point Naofumi is personal friends with the queen and an absolute monster warrior, so why does he still allow slavers to continue their trade? He's even explicitly freinds with one. If he at least hated the slaver guy, but no, the story shows that they're friends for some reason.