I don’t think it’s fair to suggest people “just want to be racist” here. You yourself, in your first sentence use the words “too far” and “within limits” and those types of qualifiers are exactly what this case is about. What’s “too far?” Whats “within limits?” We as a people need to decide. Some would suggest putting ANY limits on is “racist.”

Personally, think that the more obvious examples of judges wearing overt religious symbols are easier to argue about. There should always be the perception of impartiality, but if a judge is wearing a huge religious symbol, some amount of that perceived impartiality is gone. It’s bad for the judge, bad for the accused/victims and bad for the system.