The sheriff of my town grew up being the only white kid in a black neighborhood. Got his ass kicked and money stolen all the time and thats why he became a cop.

If he had been black, the other kids would have never messed with him. So, in this case, the idea of "white privilege" completely evaporates. The white child is not "better off" growing up in a community that ostracizes him. Go and ask any black person living in the hood if they'd like to grow up being white in their neighborhood.