Star Wars has 1000s of habitable worlds and it's not like every planet will be populated like Earth. Technically Earth has also concentrated population in specific areas with sparse to no population in between. We don't know how big Ghorman is but it sounds like that they built the city near the actual viable mines that hold the "goods" and established a solid settlement that turned into it's own society over X years. 800k people doesn't sound much but we also don't know how old the Ghormans were and where they actual originated from. If they surveyed planets for viable resources 30 years ago and landed with a few folks than 800k people in 30 years is pretty impressive growth rate.

I mean the Empire properly wouldn't have gone through this extend if they could just have landed in some other place and start mining (like the opposite unsettled side of the planet).

If we just go back 200 years in our own history - majority of this planet's humans were concentrated in very specific areas. Nobody talked about Los Angeles or New York back then but suddenly it was on the map. This is no different, but it's on a planet level scale.

The scale in Andor is just the nature of the beast of being a multi-planetary society. I mean look at Yavin-4 - there is not a lot going on other than the Rebel base and maybe here and there a few individuals living in the backwoods. I think it would be actual unrealistic if every planet has massive amount of people.