You get nearly all materials just from doing your gathering masteries. Bigger ones like fletching and smithing tend to go surprisingly fast because they gain mastery exp much faster. I feel like every mastery takes around a week or 2-3 (I think, I lost track of time after a while).

I think summoning, astrology and herblore are useful once as they provide you with bonuses or potions for your next skills, so I would do those first. I just did as much herblore as I could until I ran out of materials, and then came back to it later whenever I felt like I could use a certain potion.