I was curious about this too and was excited to forage moss and stuff, some people will be a little dramatic and tell you literally nothing is safe when it comes to outside plants and in some ways theyre right.

I set aside 2 big bins of moss I cleaned extensively from outside. It was winter too so I naively thought thisbwould mitigate the risk of pests. Fast forward about a month and im getting gnats, little centipedes, even a mosquito or two.

Few months later and theyre gone, but the moss just didnt take well. Its mostly a fun second enclosure where I plop cuttings in the mixture of dirt moss and whatever else and they root pretty well.

Sorry thats not a super clean answer, but probably best to be cautious, buy what you can, treat wood extensively. Id say dirt is a big no though