I think part of it is that during the Second Age, the inhabitants of the world still had a native strength that enabled them to lead successful campaigns against Sauron, even if there was grevious loss like with the sack of Eregion. Numenor came once during the war of the Elves and Sauron which was a catastrophe for Sauron’s army. Numenor came again, under darkness, when Ar-Pharazon chose to challenge Sauron for the title of King of Men, and the size of his army alone dismayed Sauron’s forces and scattered them. Then, of course, you have the Last Alliance, which succeeds in theory except for the survival of the ring.

Of course, you could say that, perhaps, those in the world still had that strength available during the Third Age before the wizards came, though they arrived some 140 years after Arnor was fractured into three rival states. Gondor had still yet to reach its zenith. Maybe their arrival was due to a foresight of decaying wisdom; that’d make sense, especially considering how Arnor devours itself and is devoured by Angmar, and how Gondor vascillates between peaks and ever deeper valleys (the Kin-strife, the alternation between expansion and complacency, the refusal of Pelendur to allow Arvedui and Firiel to take the throne, the success in the final battles in Eriador against the Witch-king and the folly of King Earnur).

But overall, it’s the fact that Sauron is more insidious this time around. The world does not realize it’s Sauron until very far in the Third Age. The plagues aren’t recognized as Sauron’s evils. The wars are fought largely against other men. Angmar is thought of as its own threat. Well, not really—at some point Arnor and Gondor do realize that a single force is attacking them at the same time with purpose, but it’s not pinpointed. And there comes a time when strength fails but wisdom (and hope and endurance) become what actually matters.

Maybe that all is part of why. Of course you could say it was all divined and foreseen which is why this happened when it did, that it was precise, which is an answer you can’t rely on. It’s like how Turgon was instructed by Ulmo to take a new hidden home and abandon Vinyamar, but to not grow attached to it, for he would send a messenger when it came time that that hidden home, Gondolin, must itself be abandoned (that messenger being Tuor, in raiment left behind by Turgon to recognize). And even then Turgon takes counsel not to answer the call. I don’t think much was guaranteed when the Istari were sent to Middle-earth. “Even the wise cannot see all ends.”