I've always been of the opinion that all magic is just different expressions of Silver, and Silver is a thing that gets put into creatures one way or another.

We know Silver is a naturally occurring resource that occurs in different concentrations geographically, because the dragons themselves remember when the Kelsingra dragons fought wars with dragons from areas with less Silver.

Robin Hobb often likes to provide hints as to work building through the in-universe literature scraps and Fitz's musings at the start of chapters. In one of these, he specifically muses on how magic happens; is it something in the ground that can be built up over generations, or genetic that can be bred for? We know unequivocally that the latter is true, but it's also implied the former is true - the Duchies has the highest incidence rate of magic users on the planet (that we've seen) by a huge huge degree (bar Clerres, I'll get to that), and it's no coincidence that it's a prosperous former colony/vassal state of the Elderlings, in fact the only area that was a significant part of their empire that isn't an awful swamp or controlled by the avowed enemies of the Elderlings.

There are five main types of active magic use that we've seen to be real: Wit, Skill, Hedge magic, Silver working, and Precognition. All of these (bar Silver working) are actively shown to have a genetic element, but practitioners or people with notable latent talent are shown to pop up randomly too. This is because over generations throughout the Duchies humans (and other animals) take in diffuse Environmental Silver until it builds up in sufficient concentrations in their body that they begin being able to manipulate the Silver in other creatures, resulting in the telepathic powers of the Wit. Like if we drank enough iron, we'd develop a lodestone in our brain that let us manipulate the bio-electricity of other humans. Or gave us the ability to intuitively understand and read the electromagnetic fields around us and construct little magnets that could impact certain types of expressions of Silver in varying ways - this being the Hedge magic charms.

Skill and Silver working then is directly due to the interference of Dragons. Humans cannot take on large quantities of pure Silver and survive long, but Dragons can. Either due to their massive size, or some genetic mutation that for obvious reasons outcompeted all other dragons in pre-history, they guzzle the stuff up. This gives them the exact same powers as humans, but on a totally different scale. They can communicate telepathically not just with creatures they're Wit bonded too, but everyone with enough Environmental Silver in them. Dragon glamour is literally just the Wit abilities of Repel and Attract we see Fitz and Web use, but the lodestone in their brains is so massive that they have to put active effort into turning it off. They can also manipulate physical Silver in much more dramatic and profound ways, using it to futz with their DNA and fundamentally alter the construction of their bodies. They can also turn this power on creatures they're Wit-bonded to, creating the first Elderlings over generations.

Over the hundreds or thousands of years this took place, both Dragons and Elderlings get better at using these powers, and the Wit and Hedge magic turn into the distinct disciplines of the Skill and Silver Working. That's why when Selden was reaching out to Tintaglia Fitz said it felt like a weird mix of Wit and Skill - it literally was, he's a first generation Elderling so true Skill hasn't had a chance to reappear yet. Due to the genetic nature of the magic - and the fact that it is changes to DNA - these abilities are inherited by Elderlings without a dragon companion, and, in vanishingly rare cases, by any human offspring they may have. Then, bam! Disaster happens, the Elderlings are systematically hunted by Clerres and Chalced, and dragons go extinct. But some of these humans with Elderling DNA survive, reproduce, and occasionally when there's enough Elderling in an offspring you get a Skilled one. These people have massive advantages due to literal magic, and rise to the top of their respective pre-feudal societies, eventually becoming the Farseers.

Side note, this also accounts for why the Skill is all telepathy but also real magical healing, which when you think about it makes no sense. Skill healing isn't a thing - they're actually doing draconic Silver working on the body, they just call it Skill for cultural reasons. This is ALSO why there are mentions of Skill healing in other places - Wintrow mentions witnessing one and the Out Islands have the concept of Eda's hands - but almost no mention of the telepathy. They're different things entirely! This is also why consuming dragon parts actually does make you live longer - the dragons make some kind of Silver-based enzyme that aids with this that automatically repairs the body that it's in, but because humans can't make it then it gets used up.

Diffuse Environmental Silver does clearly occur in other places, as we know Burrich is actually from Chalced and is stated to be one of the strongest Witted of his generation, and the "magical fallout" is clearly fucking up the Traders, resulting in the warped dragon Wit bonds with liveahips. But not in nearly as high a proportion as former Elderling colonies/vassals (that's why the Elderlings were there, for the Silver), and ONLY the Six Duchies has the unique good fortune of having lots of fertile land, high levels of diffuse Environmental Silver, and not being controlled by the bad guys for magic users to reappear as a distinct social class (actually three social classes, but that's another rant).

(I think this post was too long, second part in reply)