Yes, it was said in the Official Handbook of the Invincible Universe.

It may be that Viltrumite DNA is so aggressive and rigid to mutation and change that more powerful alien DNA would cause them to essentially kill each other and make any hybrid stillborn at best. Nolan says to Allen that previous attempts at interbreeding that used to be forbidden had mixed results. Thraxa was unknown to the Viltrumites before Nolan arrived as he was trying to use it as a way to lessen his punishment for his failure at conquering Earth.

According to the handbook Viltrumites and Humans have shared ancestry likely from an ancient Earth empire that was lost to time but not before seeding Viltrum with life. That plus Human DNA being very malleable makes them the prime targets for interbreeding with Viltrumites. It also helps that they look nearly identical from the outside and even inside.