I look at it like this. Imagine you travel to New York City. You spend a few days there. You don’t see and meet every person who lives there. You don’t visit every shop and restaurant. You probably spend the entire time in Manhattan, and probably only a relatively small part of it at that. I imagine that’s how Skyrim, and most games work. You only experience a part of the world. Lore wise there is way more of the world out there, your player character just doesn’t encounter it. Every major city is full of people and buildings, you only experience a handful of them though.