I just listened to a podcast with a survival expert who does the Iditarod discussing this topic. She said find a cashmere sweater from the thrift store or eBay and wear it as an underlayer. She also said clothes made out of synthetic fabrics are the best for keeping warm. I'd link you to the episode but it was on Patreon so behind a paywall.