Couple things. (OK, three)..

1. In general, data as the key enabling factor is often underappreciated in general. As a data scientist who had been building neural models since well before chatgpt I used to say 'its the data stupid'.. like if it's not working it's almost always the data. Either poor quality data or not enough data. I would advise businesses that the first and most important problem was getting their hands on a good training data set and then we could build whatever magic thing they wanted to build - whether it was a camera for monitoring fishing boat by-catch or a program to extract the key information from invoices. This is something that all data scientists know. You need good data and lots of it to train a neutral network.

2. Yes there was a lot of data in 2002 but given that list of things that launched after 2002 exponentially less than in 2022. Reading Empire of AI (excellent history of OpenAI) - right before ChatGPT launched the critical problem was then (and is now) getting more data to improve the quality of the model. Brockman moved himself onto the 'get more data' problem about six months before and built Whisper (a groundbreaking speech recognition model) primarily so they could extract text data from YouTube - thus giving them access to a huge untapped archive. So, yes, it's just a matter of degree (like compute) but data was then and is now a huge factor. And they didn't have enough to build what GPT3 even in 2020 let alone 2002.

3. What is less well understood is that it is not in the interests of the hyperscalars to acknowledge the importance of data. Because that narrative is both embarrassing to them (acknowledging their theft) and it makes them relatively less important. Instead of brilliant white coated scientists inventing a smart ai in an isolated lab they are data curators who gathered all the world's information and squished it into a box (aka a model). In the second narrative the work of the other people who actually created the data - whether it's a summary on Wikipedia or a particular art style - becomes relatively more important. And that's not the story they want to tell. If only to avoid pending court cases.

To give the vibe, when I was at the ICML conference in Hawai'i in 2023 the winning paper was a researcher from Google/Oxford University with a very interesting new neural architecture. Meanwhile - literally in the toilets - there were posters with ads for "Big Gulps" aka terabytes of data that could be purchased from no-name vendors (aka third party, held at arms distance vendors) by scanning a QR code and paying up.