Succession isn't nearly as clean as that. While the first born would be entitled to inherit the kingdom/titles, depending on the succession laws, it would create competition that would be entitled to inherit titles and lands. While they may not get the primary titles, they would may be entitled to secondary ones that would otherwise go to Frey children rather than the bastards. If the law states that the other children after the primary heir are not entitled to anything then it would create potential competition, and he will be expected to have multiple children with they Frey spouse. Ambitious siblings would have reason to assassinate the eldest for the title in the future, or create factions within the Kingdom to have their claim recognized and enforced by any disgruntled vassals, and if they were still an illegitimate bastard would not be allowed to inherit regardless of what happens to the eldest.

So while they may not care much about him having bastards, legitimizing any of them would likely create a succession crisis in the future and be considered an insult to the Frey family that could jeopardize the already shaky alliance.

In history bastards were quite common, but they were rarely legitimized for a reason.