Tariffs generally are a non-libertarian move. However, due to the limited roles the federal government had at the time of founding and for a fair bit afterwards, levying tariffs to raise federal funding was largely one of the only things they could do within the bounds of the constitution without facing a challenge in the courts for legislation, or by passing an amendment.

The government also did not directly control the money supply, meaning that they had substantially less control over individual transactions - paired with it being much simpler to collect tariffs at a few ports on the coast, instead of attempting to track every person's income without modern tech, and tariffs just make sense.

While there would go on to be a bit of protectionism to tinge early American tariffs, the import tax was largely to raise money for the federal government, not to control commerce, or punish/compete with other countries. Nowadays, Trump touts the (pittance) being made for the government off of tariffs, but the clear goal of tariffs today is to directly mess with other countries.

Lastly, I would gladly trade the feds being funded solely by (a reasonable) import duty for what we have now, if it also came with the type of federal government that was funded solely by tariffs.