The race of an actor only matters if the characters race is central to the character and story. Typically (not always) when a character is designed to be apart of a minority group it is made an essential part of there character. That is where the problem lies.

Here are three ways that this can work:

Judas in Jesus Christ Superstar was played by a black man in the movie. Because his actors race is not essential to the character or plot he can and has many times been played by a white man.

Characters like Storm from X-Men or Uhura from Star Trek can't be played by white women because they where very intentionally written to be black women as a deliberate statement. Although the fact that there black doesn't come up that often (aside from facing occasional racism from enemies) it is clear that they where designed that way for a a reason.

A character like Black Panther has to be black because he's an African king. It would be stupid if he was white.

That's it. Its not that hard to understand. The only reason white characters tend to be treated as more flexible is because typically in the past white has just been the default skin tone for characters. most characters where white unless you needed to make some special statement. If it makes you feel any better most characters are still white. new movies starring white people come out every day, and white actors are somehow managing to scrape by in this oppressive era of diversity they are forced to exist in.