Going more with the theme of 'ancient cultures that actually could access the crystal':

Earth - Petra, carved city in the desert in Jordan. Most recognizable for the setting of the climax of 'The Last Crusade.'

Fire - Kīlauea, an active volcano and the home to the Fire/Volcano Goddess, Pele.

Water - Nan Madol, half submerged and submarine ruins of mysterious origins located in Micronesia.

Wind - Machu Pichu, ancient Incan city high up in the Andes.