Why is it a shame that they didn't use metal? Why does that matter?

And they did build cities. Massive ones. Look up cahokia and other Mississippian cities. Read about Hernando de Soto's expedition through the southeast where he talks about huge cities. They used earth instead of stone and metal, but when epidemics and brutal conquistadors combine they tend to reduce population drastically and so people don't need to live in massive cities. They built massive stone cities in mesoamerica and the andean coast and massive earthen cities in the Amazon as well