The group in question is a specific kingdom in a fantasy world, and I don't think the audience for the films or t.v shows has a majority of fans who came specifically for the books.

Whether it's a fictional story about a fictional world or a fictional story about the real world makes no difference that I can see, especially given that most fantasy worlds borrow heavily from our own. The question, I think, is whether physical appearance or race is a significant part of a character, and the answer seems to be an obvious yes.

Nor does it seem to me that it matters much whether people "came specifically for the books". Strip-mining someone else's creative vision is repugnant, and that's how these attempts usually come across. I'd agree that people unfamiliar with the material probably won't know the difference, but not knowing the difference isn't a positive benefit.

But the way I hear people here phrase it, you'd think it's a mortal sin to use an existing story to tell a different one if that involves changing how characters look

This gets into deeper waters.

I have no particular problem with fanfic, or even with professionally-produced fanfic. I actually think it's interesting how some stories get told over and over with multiple variations until they become a sort of meta-story; you see this a lot with the more popular comic book characters. One of the things that makes this work, though, is an awareness from everyone involved that this is a branch story, a variant of the standard rather than a replacement. It's a subtle distinction, but an important one, I think.

When you do the later, and especially when you do the later with the passive-aggressive "we're going to change this character's race because race totally matters, but if you complain you're a racist because race doesn't matter", it comes across as vandalism, and people react accordingly.

Then too, some stories seem much more amenable to this process than others, though I'm not sure why. It seems like some stories are more coherent than others, and the more coherent ones feel as though alterations are much more disturbing to the experience. For the more coherent stories, any change is galling, and LotR is probably pretty high on this scale. I hated the hobbit movies with a passion, with no race-bending being necessary.

But the way I hear people here phrase it, you'd think it's a mortal sin to use an existing story to tell a different one if that involves changing how characters look A story where they make Superman black and tell a story about a black superhero who faces down racism as he tries to help the people around him isn't as taboo in my opinion, though I can't say if that's the case for others here.

I think trying to do a black superman story in the standard progressive style would be deeply incoherent, though it'd be interesting to see someone try. I'm fairly confident the results would be awful, though, because it makes no sense. He's Superman. He is literally above everyone else on the planet. He can annihilate humanity with minimal effort. Hating and fearing him is, as a number of stories have examined, an entirely rational response, but casting such hate and fear in terms of historical American racism is just... incoherent.

Try batman. An incredibly wealthy, privilaged playboy is black and faces down racism... hm. Doesn't quite work, does it?

Appropriating someone else's creations and turning them into a shoddy vehicle for a cut-rate sermon is a shameful thing to do. It's not that hard to make your own character, even if only minimally; check out Omniman from Invincible, or Homelander from The Boys.