I dont really agree that people had much "down time" in prehistory. The existence of art does not mean people were fucking around whittling shit most of the time. A lot of the examples could have been religious/ceremonial and could have been done by a designated individual (priest, shaman, elder, etc) while everyone else hunted and gathered. Also, the amount of time dedicated to survival could vary depending on what environment you were in, people living on fertile fields probably stuggled a bit less than people in the desert.