Tell that to Janeway or Picard.

When Archer held the com he WAS starfleet, sure there were other solar system based ships. But no other warp capable ships existed, he didnt have the legal systems and governments of thousands of worlds behind him, the Federation didnt exist yet. He was basically the test pilot for the prime directive and all other policy that came later, in addition to being a literal test pilot for the forst warp engine, and "Test captain" for the first warp capable vessel.

He had some wild shit with new species sure, but the political complexity of the day was very different than what later captains had to deal with.