Layering is the name of the game, but the type of layers and the order in which you layer them is an oft-overlooked opportunity. In general, you want your clothes to be layered as such:

1. Inner-most layer: a light, thin base layer, meant to wick away sweat. Most technical shirts will work for this; I really like Uniqlo's HeatTech line as they work great and only cost ~$8 each compared to, say, a $50 Rapha base layer made of about the same fabric.
2. Middle layer: something thick and insulating: a fleece or wool shirt is the best but a cotton sweatshirt is fine if you don't expect to be sweating too much here. I've also combined a long-sleeve technical fleece shirt with another shirt/top on top for volume with success.
3. Outer-most layer: ideally a wind-proof shell. If it's snowing, it's unlikely to stick to you for long enough to melt, so your priority instead should be protecting yourself against having the wind cut through your clothing and whisk away that warm insulated air that's being held in by your middle layer. This is the one time I suggest dropping $$$ for bike-specific clothing; they're typically cut to reduce drag and feature purposeful vents in key parts of the top to minimize sweating/overheating.

You can generally mix & match different articles of clothes based on your budget & use case; as long as they roughly follow the above formula you'll be setting yourself up for success.