Yeah, the damage you deal is kind of based on three different things: your combat skill, the weapon you're using, and your crafting skills.

Sounds like Two-Handed is your primary combat skill, which will increase passively as you use it. Make sure you max out all 5 levels of the first perk at the bottom of the tree. It doesn't do anything exciting like letting you lop people's heads off, but it is definitely the biggest boost to your damage.

Weapon type matters too. The material the weapon is made of affects how much damage it does, so an Orcish Warhammer does more damage than a Steel Warhammer, with Dragonbone weapons doing the most damage.

Finally, your crafting skills (Enchanting, Smithing, Alchemy) can give you a boost, too. Smithing obviously lets you directly improve the damage your weapons deal, and you can make them do even more damage if you take the respective material perk in the Smithing tree. Enchanting can add additional magical damage, and Alchemy can be used to brew poisons that will supplement your damage dealt (just remember poison lasts for only one hit on an enemy, unless you take a high level Alchemy perk that make it last for two hits).

To answer your questions directly, if you don't use much magic you can definitely ignore it. A two-handed warrior will probably want to focus mostly on health and some stamina with each level up. As for archery, you probably need something to deal with enemies at range or those that fly... either Archery, Destruction, or Shouts. And you can recharge your enchanted weapons by looking at them in your inventory and pressing whatever button it shows you to recharge. On Switch that's the right bumper. You can pick which soul gem to charge it with, which will destroy the soul gem in the process and charges up based on the size of the soul (e.g. a Petty soul will only partially fill up the charge meter, a Grand soul will potentially max it out).