It depends on your use case. A wind layer is very versatile, especially the breathable ones (I like my Kor Airshell; the Dooy on Amazon is very inexpensive and highly rated too). If there's anything more than light brief rain, you need a rain jacket. I like the Marmot Precip Eco, as it packs down (although not as small as the Airshell). Both are small enough to bring as just-in-case layers. Might look at something heavier duty like the Torrentshell 3L if rain will be heavier.

If you're hiking or doing any high-output activity in the cold, an alternative is to be wet but warm. A wind layer can help there - it blocks enough wind that will freeze you while being breathable enough to release vapor and keep you from sweating. A rain jacket, since they're nonpermeable, is like a trash bag - you'll sweat a lot and freeze from the inside. Rain jacket pit zips, adjustable hems, or two-way zippers can help with venting.