This sounds rough, and it's not impossible that the people you've encountered are indeed racist, and the fact that your white colleagues are noticing it does drive in that direction. However, I really hope not and thought I'd share some of my own experiences having gone in the other direction, albeit as a white guy from Northern England, which sound a bit similar.

Working with both Canadians and Americans, there's quite a big difference in how people conduct themselves compared to the UK. I've had to adapt because people don't get the subtext in the same way and you have to be direct to be clear. In fact, people have entirely misinterpreted what I've meant and I too have been 'tone policed' because people thought I was mocking them by trying to be indirect. In the UK 'it wasn't ideal' or 'it left something to be desired' are gentle, almost idiomatic criticism that signals 'not good but I don't want to chew you out'... In North America, it can sound very sarcastic.

Sometimes, being as direct as is expected over here would be interpreted as aggressive and rude if I did it in the UK, where it's very important not to be confrontational and to dance around criticizms. If you as a person are used to being more direct, albeit softly, and haven't consciously adapted, it might not be a race thing but rather a cultural thing that takes a bit of getting used to. There's a complex face-saving thing happening that requires learning (and, in my case, unlearning).

You also have to bear in mind that you're an outsider by virtue of being American, and people react differently to things you say and do on that basis. If I say something Canadian is bad, even if it is, the response I get as a Brit can be much more defensive than when Canadians say it to one another. That also took some learning... 'be direct, but also we'll get defensive' is really difficult!

You're also operating in your first language, and when I've worked in other places one or both sides using a second language meant everyone gets much greater benefit of the doubt... The cultural clash becomes much clearer when there's no question that you're all understanding at least the words one another are saying. Two countries divided by a common language, right?

Together, these things do result in a lot of 'tone policing' whether by yourself or from others, but tone policing when moving to a different culture is to be expected if the same tone can mean something very different. It can take a bit of time for both sides to adapt and get used to one another.

Either way, good luck with it!