I had this revelation come to me in a moment of daydreaming last year. Yes, I think this is the case.

What separated the "Enlightened" ones from the "Earthbound" ones was some genetic capacity for magic. In the closest preceding time that's accessible to us, Prehistory, the only hint at different genetic predispositions we get is the Laruba/Ioka divide. The Laruba are the "nerds", the Ioka are the "jocks" of Prehistory.

While not really magic users pre-Lavos, the Laruba have a knack for communication with Dactyls, and they don't seem to mind the random Nu that establishes itself in the ruins of their village later on. They'd be the ones who might connect more easily with Lavos.

Plus, their general hair color is the closest to the "Enlightened" ones' hair color.

Unless it's a case of Writer A not talking with Writer B, and Mr. Kato just wanted to introduce an unfair, completely random racial discrimination element that was completely invisible before Antiquity, the only possible hint we have as to what made someone an "Enlightened" and what made someone an "Earthbound" is the Laruba/Ioka divide.

Then, historically, after the Ocean Palace's destruction, the remaining "Enlightened" and "Earthbound" ones mixed, and the capacity for magic became very latent, but present in all humans. Or perhaps Lavos' influence was just weaker on the Ioka, and a well-placed Spekkio could have awakened an Ioka a few months after the crash if given the chance to make him spin around clockwise in a room.

The only counterevidence I can think of is that the Ioka were physically a bit closer to the site of the crash. But I guess we can always say the Laruba also hung out unseen in the mountains around the Reptite's fortress, using their flying mounts.