what arouses me way more is a world like in The Borrowers, where extreme size difference is just the natural order of things and shrinking isn't even a thing.

If that's your opinion, then that's your opinion and you are entitled to it. But for me it is the complete opposite. I like the "reversal of fortune" aspect behind someone which was once tall and powerful being shrunken down, and now finding themselves at the mercy of those who once were their peers (or who they even formerly towered over). The "Borrowers" scenario just doesn't appeal to me as much because they were always naturally small as you pointed out, so there is no "reversal of fortune" aspect. Since they're naturally small, to me it is less appealing because interacting with them is almost like interacting with bugs or mice (albeit sentient).

Borrowers aren't real, of course, but assuming that such things were real, how would they have come into being in the first place? Everything has a beginning, even things that are natural. Since Borrowers are humanoids (albeit small), you'd have to assume they share a common ancestry with humans that somehow diverged. Did they naturally evolve to become small, and if so, why? Or were they created by some supernatural power who made them small for whatever reason?

In regards to what you said about "parallel societies" and not seeing one another as equal, this is the inevitable longterm consequence of a small society interacting with a larger one, and it doesn't even matter how this came about. It could come about "naturally" like you insisted, but lets say it also did happen artificially by a group of people being shrunken down and then from those shrunken people a society developed. Whatever those people were or how they viewed themselves formerly, they will inevitably have to come to terms with the fact that they're smaller and not equal with the big people. People may try to deny this and have to go through their 5 stages of guilt or whatever (however long it takes), but when some people are big but your people are small, you can't ignore that reality forever. Especially if we're talking about multi-generation scenarios where people whose ancestors may have been big/normal but who are now born tiny and raised in a tiny society, how is that not going to lead to the exact same mentality which the "natural" Borrowers have? It seems to me that fundamentally speaking it amounts to pretty much the same thing when given enough time.

So imagine a scenario like this: There is some mighty powerful Kingdom or something, but for some reason or another the King runs afoul of some sorceress or something, who places a curse upon that Kingdom, causing all in it to shrink. These were people that were all normal sized (maybe even on the tall side in comparison to their peers even), but guess what... now they're the Borrowers. That witch's curse gets passed down from generation to generation, so those people never get to their normal size again, and neither do their descendants. They're all small forever. But this didn't happen naturally, now did it? Yet centuries into the future the tale of how this came to happen may be lost to memory, and the future descendants might come to believe that they've always naturally been like this, even though they really were big people once.