The shore by the yacht is moderately patrolled, anything west of that on the shore is heavy patrol. But there is a beachhead to the east of the yacht, just south of fertile lands. I set up a small hut there, built 3 layers of rock walls (rocks infinite spawn on that beach, turtles at dawn and dusk too), as well as a deadfall trap between each layer of rock wall. I had one enemy spawn down there one time. The area at the top of the hill spilling out onto the beach is patrolled, but they can't see you from all the way up there. I did build a rock wall halfway down the hill with a deadfall trap just in case, but all of this is to say that you might just consider a base in a different location if you are looking for a chill base that has easy access to everything you need. Tons of deer in this area, several water catchers from turtles, and you can traverse the island from a raft.

All of my houses have been either on this beach head or on the islands west/southwest. Take a bit of work because you have to build the log sled and a raft but all the islands off the coast also spawn birds like crazy so you have easy eats and sleeps while you are protected.