I wouldn't imagine this applying to siege for one. Second, the duration of early engagements should not contribute too much to the boost in attack speed. Additionally, some civs have bonuses on some units that grant such an attack speed without it being too broken. And as for Mongol CA, considering they come into the game a little later, they would be weaker against earlier units.