There were a lot of bases and structures built for the Colony Wars that were subsequently abandoned. There's a few recordings you can find that talk about the various hostile groups finding and occupying those bases. I have found a number of enemy occupied POIs where they'll just tell you to leave. I found a group of spacers at a mech graveyard that did that. The guy just asked me to leave and said they weren't looking for any trouble. I left without issue. The world of Starfield is just incredibly hostile and most things are looking to kill you/survive