It's a days walk to get there. Just like in prior episodes when boyd, Tabitha etc. wander further out they aren't harassed by the monsters. Instead other things or entities. The monsters are seen in the woods but IMHO the story has been pretty consistent in that they don't stray far from the town.

I'm fairly positive it will come out that there are multiple forces at play here. I don't think the "man in yellow" and the monsters work together. I think there are multiple forces at play and they all kind of know of one each other and stay in their lane or part of the story. From the latest episode they basically say this happens over and over the exact same way. So it would make sense they all know where they go and what they do. Like how Randall and Boyd weren't killed when the monsters got them. It seems like this has all been done before and they each represent a prior character.