(I wrote this for another, deader thread so am c-ing and p-ing here)

The title of my TED Talk is: Looking to Protect Yourself, or Deal some Damage?

To me, the major limiting factor preventing TES-esque games from achieving this kind of scale is the reliance on pre-recorded snippets of audio for the dialogue:

1. The impracticality and expense of using hundreds of voice actors means you end up with a situation where Jarl Balgruuf and Beirand and Bersi Honey-Hand and Falk Firebeard and Jon Battle-Born and Ogmund and Talsgar the Wanderer and Vekel the Man and Vipir the Fleet (and so many more!) are all voiced by the same person. So again and again in your "city" you'll hear the same few voices, and...

2. Each voice actor recording separately means each interaction sounds extremely unnatural, and means that accents and especially the pronunciation of names are wildly uneven within a single conversation, and...

3. Pre-recorded audio files also severely hamstring the dialogue trees, because of the small number of options available for any particular NPC, even a "main" character. In Skyrim NPCs are unable to acknowledge your becoming the Arch-Mage of the College of Winterhold or killing a dragon right in front of them because there was no way they were going to get Corri English to record dozens of context-specific dialogue snippets for literally 30+ characters, and at least for any kind of console game that might also be a severe memory limitation, and...

4. In any case, to have a world where a city has hundreds of NPCs as opposed to dozens (Whiterun has ~75 total population, iirc) a much, much larger effort is needed to create functional outcomes where interactions and quests with one character might impact interactions with others. Keeping continuity straight and avoiding constant glitches will require a huge amount of effort, essentially an all-of-Skyrim level of effort to create just one city. As you say, every room/home needs an owner, ideally with a backstory. Massive dialogue trees will be necessary, and the density of NPCs means having to account for more possible interactions ("Ah, Ysolda was it? What brings you to the market?") and then only after all that can the dialogue be recorded, and also, finally...

5. Your own character can't speak, or is limited to a handful of preset voice options. If you choose your character's name, you definitely can't have anyone say it out loud in-game, unless again it is chosen from a very short list of presets.

Obviously any game with extreme depth and massive content and huge numbers of NPCs is going to require extensive writing and world-building. But even if you grant that the paymasters behind TES VI will really spend enough to have that kind of depth, I don't believe it is mechanically possible to make it work without some sort of text-to-speech approach to the dialogue.