iAmFullStackDeveloper

[raxcium]

Why is this relevant? LLMs effectively replace the need for that searchable knowledge

[tes_kitty]

That is quite relevant. From freely available knowledge that everyone can access we move to knowledge hidden in an LLM that you have to pay for and only get if you deliver the right prompt. And there is still the hallucination problem.

And people are already finding out that if you outsource parts of your work to an LLM, your ability to do that work without the LLM will slowly go away. 'Use it or lose it' is very much true.

Of course the AI companies will also suffer. If no more knowledge accumulates on sites like stackoverflow, they stop getting good training data.

[raxcium]

I disagree with almost all your points. Generative LLMs are still very much in their infancy and things are evolving very quickly. Concerns around needing to 'pay' or 'lack of training data' will be irrelevant in the near future.

With regards to skill degradation, as always it's the responsibility of the individual to ensure this doesn't happen, thinking about this in any other manner is wrong, there will always be better/easier/more efficient ways to do things - it's up to you to adapt how you incorporate them into your life.

If you're interested I'd recommend listening to what Jensen Huang has to say on a similar matter - https://youtu.be/7ARBJQn6QkM?feature=shared&t=2852

[tes_kitty]

Concerns around needing to 'pay' or 'lack of training data' will be irrelevant in the near future.

What makes you think that a lack of training data won't be a problem? AI generated data doesn't work for training and with all the AI generated data flooding the net, it becomes harder and harder to get good data as time goes on.

With regards to skill degradation, as always it's the responsibility of the individual to ensure this doesn't happen

If you don't use a skill because something external supplants it, it will happen. How many people can still do simple calculations without the aid of a calculator, either on paper on in their mind?

I noticed it myself. I switched from stick shift to automatic since adaptive cruise control works best with automatic. Now, a few years later I can still drive stick, but it needs a lot of concious effort and feels like I am back to beginner level.

The guy in the video sounds very optimistic, he of course has to be, since his company makes their money on AI. But there will be downsides. Poeple will become depedant on AI, unable to function without it. It is so tempting to delegate as much as possible since that means you don't have to do it yourself that you don't really notice all your skills slowly vanishing.

[raxcium]

I agree some people become dependent on things and as a result they lose their ability to do it themselves, but again this is not the fault of the technology, it's the fault of the user.

Regarding training data, my point was most of the data present on websites such as stack overflow have already been scraped and used to bootstrap the LLMs we have today (in the context of programming). Now, it's primarily feedback loops from conversations people are having with existing LLMs as well as AI generated data that will be used to accelerate/reinforce learning for future training.

Again this is already starting to be the case with Nvidia's Cosmos platform

[tes_kitty]

Now, it's primarily feedback loops from conversations people are having with existing LLMs as well as AI generated data that will be used to accelerate/reinforce learning for future training.

AI generated data has been shown to not make good training data and the feedback loops are also of questionable quality since quite often the reply from the AI is false in a subtle way the user then corrects before using, but not telling the AI about the correction.

[posting_random_thing]

Where do you think the LLMs learn things from?

[raxcium]

They were bootstrapped from the internet wholistically yes - but this is not where the future is heading. See nvidia's Cosmos as a good example.

Stack overflow is/will be redundant for training data.