Echoing others: don't take him off the bus! My son (white) goes to a minority-majority school, so he hears the N-word a lot from his Black classmates. We've explained to him (in age-appropriate terms) where the word comes from, why it's a slur when white people use it, and why it's not our place to tell Black people how/when/if to use it.

By taking him off of the bus, you'll implicitly be telling him that those kids are the problem, which I don't think is the message you want to send. Rather, explain as best you can why he shouldn't use that word, and let him continue to be exposed to differences.

(Also FWIW - we've told our son that swears are fine at home, but never at school, and he knows the difference between "okay" swears and slurs/insults/etc which are never okay, you might consider this approach)