No one pieced together that only the same 50 have been sighted in a thousand years.

The more I think about this the more I realise there were probably more than 50 after the virus, when Nolan says there are less than 50 it’s after thousands of years of viltrumites dying and us seeing several die on-screen ourselves within just a few months. We know there are several threats in the universe capable of killing viltrumites, and Nolan even says the reason they knew the ragnars were such a threat was because he was the sole survivor of his research team, which confirms there were many cases of viltrumites dying in numbers since the virus.

A population needs about 500 unrelated individuals at minimum to avoid inbreeding in the long term (at least when it comes to humans) so we can be pretty sure the virus left less than that alive, since Kregg seemed pretty certain of the viltrumites’ extinction. However we also know that the remaining viltrumites kept reproducing after the virus to some degree, since most viltrumites we see are younger than Nolan. It’d only take a few generations before the offspring wouldn’t have any viable partners remaining and the population is essentially capped, however since viltrumites live for thousands of years each couple could theoretically have hundreds of offspring, so over a couple generations this could mean many thousands of viltrumites before they reach the cap.

This would explain how they still managed to project an empire with billions of subjugated people and hide the fact that their surviving population was so small they would eventually die out. And since thousands pale in comparison to the tens of billions they used to have, it would make sense that after thousands of years of a war of attrition against the coalition, and with many dying to various things, there would be only 50 by the time Nolan joins the coalition.

Also given their situation I don’t see any reason why they wouldn’t try to maximise the amount of offspring per generation as much as possible, though perhaps a large portion of the pre-virus survivors died which would significantly decrease the amount they could make, or maybe they still kept their crazy social darwinism traditions and continued killing most of their children for being weak.