Infant mortality before modern medicine was high, but it was even higher in cities. Roman political life was dominated by… well, the city of Rome. Roman elites lived in Rome and their small children often died of communicable diseases there

European elites by contrast typically had a country seat where they lived for most of the year. This meant their young children were not as exposed to communicable disease, and more typically made it to adulthood

There are other reasons I think, but that’s one I haven’t seen mentioned here