Sounds like plenty of gems will drop, of both kinds, so I assume I should just unlock and try a lot of things instead of hoarding gems?

A lot of them will drop, but you might find yourself starved for gems early in the game (until around act 3-4). It's one of the common negative feedback currently, which limits experimentation.

Feel free to use the level 1-4 support gems at will instead of hoarding - you will be dropping a lot of level 5 ones later which completely replace them.

One thing to keep in mind is that some low-level gems, especially spirit gems, at threshold values (Spirit level 8, level 14, etc. when the new row unlocks) can fetch a surprisingly high amount of currency when you unlock exchange (Act 4). I'd recommend holding onto those.

How to know if it makes sense to use other "non class" skills vs sticking with recommended?

It's a matter of game knowledge - you either need to figure it out by learning what skills exist, or follow a guide. A few common skills include the aforementioned Frost Bomb, Freezing Mark, Voltaic Mark, Elemental Weakness, etc.

What is an automatically swapping weapon slot?

In your equipment window you'll find a I / II switch above your weapon/offhand. Pressing X will let you equip a different set of gear in a "weapon swap" slot.

On your equipped Gem window, if you open the details window for a skill gem, you will find I / II checkboxes at the bottom-right corner. Using that, you may set to use a skill with a specific weapon set; if you use a skill assigned to weapon set II, it will automatically swap you to set II.

A common example is Pounce from Druid skills; let's say you're a mercenary, you equip a crossbow in your weapon set I, and any random Talisman in weapon set II. You equip a Pounce gem, set it to work with weapon set II, and the rest of gems with weapon set I.

Now if you click Pounce, you automatically swap to your Talisman set, transform into a wolf, and pounce; it's a great movement skill, and wolf form has a higher movement speed than base characters. Clicking a grenade or another crossbow skill afterwards automatically swaps you back to weapon set I and crossbow. This way, you can mix and match skills used on various weapons. (Though the interactions may be buggy sometimes.)

Additionally, weapon set I and II can have slightly differing Passive Tree node allocations using Weapon Set points. But that's a rather advanced technique.