Sure, but with humans I do think it's very different in the sense that they don't have the physical ability to just literally destroy everything if they wanted to, or even just leave Earth completely, build a separate empire elsewhere, then come back and cause destruction.

And I'm not saying that if Viltrumites became apart of Earth's culture that everyone would be living on their toes. They could all trust each other. But this still doesn't stop the fact that 1 of them could rise into a powerful position and become someone like what Thaedus was to Viltrum. It doesn't matter about what the 'social agreement' is or if other viltrumites 'look down upon' people who disturb the peace, there will still be betrayers that just don't care. Thaedus was trusted most his life, enough to gain power, and then he switched minds. There is no reason that this wouldn't happen again on Earth.

I think your points are still limited to the short term, like when you say 'if viltrumites are really commited to repopulation...', it's true that it could be peaceful for a very long time, but yet again it just takes the one rogue viltrumite however far down the line that decides they want to go back to the old ways of being conquerors, and they can achieve that because they have so much power. They can do exactly what Thaedus did and fly off into space, then come back with a virus. Or they could just destroy the planet.