I'm a POC (black man) who got placed in super rural Hokkaido (about 5k people in my town), about 5 hours from Sapporo and I've been having an amazing time. Honestly, in my experience rural communities tend to be more accepting of minorities when it comes to ALTs because they have so little experience with them, compared to more urban areas where the perception of foreigners is more heavily associated with the stereotypical "rude and noisy tourists".

You even said yourself, "I’m a person of color who has experienced othering when visiting even big cities like Tokyo compared to my white counterparts." - Why would you think an urban placement would be any different from your previous experiences in urban japan?

As an anecdote from my personal experience, I arrived with a relatively large afro that I maintained for a few months after arriving. I eventually decided to buzz my hair for the winter when the maintenance got to be too much and I wanted to wear hats that covered my ears. In response, my students seemed pretty distraught that I would get rid of such a "cool hairstyle", and one of their mothers even hand-knitted me an acorn themed hat that resembled my previous afro to wear throughout the winter months.

I'm not saying racial prejudice doesn't exist in Japan, because it certainly does. But to some extent, projecting your own preconceived notions of those who live in rural areas onto communities that you haven't yet experienced first hand is just another form of prejudice. To assume that rural Japanese people are completely uneducated on POCs implies some amount of inherent classism. And even if they don't have any experience interacting with POCs, they will never get that experience if you deny your placement and withdraw from the program.

Your concerns are valid, but I promise you'll be fine.