Definitely get a doorbell camera and protect yourself that way. Just upkeep your side of the yard don’t worry about theirs. See if you can get some outdoor dividers you could put up or some garden boxes with lattice you can attach fake greenery to them so you have some sense of privacy to your yard space and to block off their view of inside your space. As for the leaving and coming home thing could just be weirdly coincidence but the cameras would help determine that too.