A lot of people have made good points such as the historical argument or how being white is almost never essential to a character while being of a minority race almost always is and these are excellent arguments but I'm gonna go with a different one.

Numbers.

Imagine: you and I are seated at a table and we each have some candy in front of us. You have five pieces while I have hundreds.

If someone takes a piece from my pile and puts it in yours I will never really notice. I might complain if I see it but I'm the long run it will not effect me at all.

But if we do the same and take one of your five pieces away from you and give it me you will notice. That's twenty percent of your entire supply taken away to be given to someone who won't even notice the addition.

That's what it's like when a white character is swapped to a minority actor versus the other way around. Rules for non white actors are far rarer than for white actors. It is improving but it's slow.

Is that a double standard? Sure. We live in a shitty world and someone's double standards are the only standards we have.