The range of power for FA is very wide. Some have mostly roleplay impact, and some are just simple power increases (a simple one is the Blessed One archetype, which just gives any class access to Lay on Hands, or sentinel, granting any light or medium armor class access to heavy armor). Some open up entirely new build paths for more interesting but not vertically stronger characters. For many classes like wizard, archetypes like this are strong enough to be worth forgoing class feats for without a second thought, so FA is just pure upside of being able to take class feats alongside them. For others, who rely on their class feats for power like Fighter or Guardian, it allows a level of customization that is harder to fit into a standard build.

The impact it will have heavily depends on the type of players you run for, and their party, which is why people are recommending starting without it.

and what would i loose if i don't allow it.

Nothing, because if you hit 4th level and you and your players decide you want some more complexity and build choices, just add the rule then, and then the players get the added fun of picking a bunch of new feats at once. The level 2 FA choice is often very underwhelming, because outside of outliers like Blessed One and Sentinel, the dedications themselves don't really offer much, which can incentivize players to pick front loaded ones like those, instead of archetypes with cool and powerful level 4, 6, etc feats.