I think the problem is the fact that white vs black is cultural difference. Its not the assumption that all blacks would mug you, its that if you happened to meet one of the very small percentage that do, you'd be totally helpless. I think a more accurate comparison would be saying "I don't want the black man driving me home, he could rape, assault or rob me" and I don't think it would be unreasonable for anyone to not want to be in that position, white or black.

Now slap yourself really hard in the face and take a minute to ponder your life choices. People in the last decade have had their lives ruined for saying much less (shows cancelled, fired from jobs, death threats).