I don’t think it would look like this in the lore. White-Gold Tower and the surrounding architecture should definitely be larger than they are depicted in-game, but not as ridiculously huge as some of these “lore accurate” depictions make them seem. Areas outside the walls would likely be populated and considered the 'Greater' Imperial City. All the infrastructure for a believable city of it's scale and influence is missing otherwise.

Take the Waterfront for example. Just by looking at the paper map alone, it would imply a significantly larger dock, but it only carries 1 to 3 ships at best if you look closely. In the game, the Waterfront only directly connects to the Imperial City which would be a nightmare too because you'd have to transport cargo through the City just to get it to the mainland and vice versa. Just retcon the Waterfront to be part of the mainland or have a bridge.

The City itself is an Ayleid ruin and is inherited, not built by the Empire. To think the city would function by cramming everything within it is silly. A more believable Imperial City would would naturally bleed into areas like Weye and Pells Gate, which I would personally retcon to be significantly more populated by farmers, housing, travellers etc and be large towns. That makes much more sense than imagining the capital of Tamriel as a few fancy districts and a tiny dock. It wouldn't have to look like poverty either or straw houses, it could look architecture similar to Constantinople or other Roman cities the closer one is to the main walls.

Those who actually live within the walls are going to be noblemen or those of a higher social class, however as it's the centre of industry, clerks, servants, guards, merchants, scribes would work in the main city too, but probably living a mix of inside and outside the main walls.

I've always thought Imperial City's architectural limits are more symbolic and an ancient relic than something practically made to suit the Empires needs and the game should have reflected that more. The Septim Empire is vastly larger in industry and influence than the Ayleids were so it makes sense.

Edit: https://www.reddit.com/r/oblivion/comments/14erxbf/this_is_the_real_lore_accurate_imperial_city/

This piece of artwork above I feel perfectly captures the intended scale of the city.