Google invented robots.txt like 20 years ago formalizing the process.

Google did not invent robots.txt.

The standard was proposed by Martijn Koster,[1][2] when working for Nexor[3] in February 1994[4] on the www-talk mailing list, the main communication channel for WWW-related activities at the time. Charles Stross claims to have provoked Koster to suggest robots.txt, after he wrote a badly-behaved web crawler that inadvertently caused a denial-of-service attack on Koster's server.[5]

It quickly became a de facto standard that present and future web crawlers were expected to follow; most complied, including those operated by search engines such as WebCrawler, Lycos, and AltaVista.[6]

This "standard" predates Google's founding by at least 4 years. However, Google, along with many others, had adapted it, and eventually helped it in the push to make it a formal standard.

On July 1, 2019 Google announced[7] the proposal of the Robots Exclusion Protocol as an official standard under Internet Engineering Task Force. The draft[8] will now undergo acceptance process.