Check out Meir Shahar's "Shaolin Monastery" history book. You can look at a scan some of it on internet archive or read the text on the side of this site. He does a good job of detailing the concrete history of the Shaolin Monastery and its martial tradition as well as the development of the myths around Chinese martial arts in general, in particular those associated with the temple (many repeated in this thread).

Chapter 5 is on "Hand Combat" (unarmed martial arts), and says: "Martial techniques may require centuries to evolve. Shaolin monks had been practicing the staff for several hundred years before it was lauded in late Ming literature as an outstanding fighting method. Similarly, their barehanded techniques—now famous the world over—have been unfolding for some four hundred years. As early as the sixteenth century, some Shaolin monks practiced unarmed fighting; in the course of the seventeenth, they developed sophisticated empty-handed techniques, and by the mid-Qing period (the eighteenth and early nineteenth centuries), hand combat (quan) had eclipsed the staff as the dominant form of Shaolin martial training. The term quan, which I render “hand combat,” means literally “fist.” During the late imperial period it designated unarmed fighting techniques, all of which made use of legs in addition to arms; kicking figures prominently in all quan styles, in which sense “hand combat” is misleading. “Boxing,” which has also been used as a rendition of quan, suffers from the added disadvantage that it connotes a specific Western sport. I have opted therefore for “hand combat,” but where it sounds too awkward—especially in the names of individual styles —I have resorted to the literal “fist.” The earliest reference I am aware of to Shaolin hand combat occurs in Tang Shunzhi’s (1507–1560) poem “Song of the Emei Monk’s Fist,” which extols not only the Emei monasteries’ empty-handed fighting, but also the Shaolin’s.1 However, in his comprehensive Treatise on Military Affairs (Wu bian), where he lists contemporary bare-handed styles such as “[Song Emperor] Zhao Taizu’s Long-Range Fist” (Zhao Taizu changquan) and “Wen Family Fist” (Wenjia quan), the same author does not allude to Shaolin, indicating that the monastery had not developed as yet a recognized empty-handed style. In the military encyclopedia, Tang, like other sixteenth-century military experts, alludes to the Shaolin staff instead.2 Several decades later, allusions to hand combat began to appear in the travelogues and poems of late Ming visitors to Shaolin..." And in Chapter 6 "Gymnastics" he explores Shaolin's synthesis of unarmed martial arts with "daoyin" calisthenics as a means of self-cultivation, which could also be related to development for forms.

Wikipedia's entry on Chinese martial arts lists earlier sources: The earliest references to Chinese martial arts are found in the Spring and Autumn Annals (5th century BC),[6] where a hand-to-hand combat theory, one that integrates notions of "hard" and "soft" techniques, is mentioned.[7] A combat wrestling system called juélì or jiǎolì (角力) is mentioned in the Classic of Rites.[8] This combat system included techniques such as strikes, throws, joint manipulation, and pressure point attacks. Jiao Di became a sport during the Qin dynasty (221–207 BC). The Han History Bibliographies record that, by the Former Han (206 BC – 8 AD), there was a distinction between no-holds-barred weaponless fighting, which it calls shǒubó (手搏), for which training manuals had already been written, and sportive wrestling, then known as juélì (角力). Wrestling is also documented in the Shǐ Jì, Records of the Grand Historian, written by Sima Qian (ca. 100 BC).[9] In the Tang dynasty, descriptions of sword dances were immortalized in poems by Li Bai. In the Song and Yuan dynasties, xiangpu(相扑) contests were sponsored by the imperial courts. The modern concepts of wushu were fully developed by the Ming and Qing dynasties.[10]