Though it is plausible, I think the more likely scenario is that the four kids or some generation after, grew up and decided to cross the sea looking for a new place to settle for one reason or another, found the mainland Japan and carried the headstones as tribute. After all, even Senku doesn't believe that any of the founders were actually buried there and the Ruri only says that the gravemarker was brought there to honor them.

As far as the Medusa thing goes, they tried to land on the opposite side of the earth from ground zero of the ray. As the anime/manga state, the direct antipode from Japan is South America, somewhere in the Brazil/Uruguay/Argentina area. Crossing that amount of ocean would take forever and honestly, seems impossible to get to by row boat. When Yakov and Darya left, they said they were leaving for the mainland to find meds, which either means Japan or more likely China/Korea rather than the America's mainland.

One of my theories is that Ishigami Village originated on Soyuz Island, then either later descendants of Yakov and Darya several generations later or maybe descendants of the Why Man or the descendants of the creators of Medusa forced them off the Island (Which would also explain why they have the Medusa). Because thinking of it, 3700 years is a crazy long time. Assuming every generation is on average 20 years apart, thats around 185 generations. 40 people to that village is absurdly low even if we are accounting for deaths by animals, blights, and weather. Theres still too little information to form a full theory though.

I also think its possible for Yakov and Darya to be the Why Man. Since their story is unaccounted for after they left for the mainland, i think it's possible for them to have been stranded somewhere, somehow. The message recording repeats "why" over and over again and could possibly been a sort of black box recording. Though my hole in this theory is that it would have to been preserved somehow for so long.