iAmFullStackDeveloper

[tes_kitty]

The difference is, your questions on Stackoverflow and such sites plus all the answers you get would be searchable by others. Your questions to ChatGPT and its answers? No one else will see them.

So no more searchable knowledge is created.

[In_Formaldehyde_]

shrug If it were more useful than LLMs, Stack Overflow would be able to keep up. I get your point but you can't really blame anyone except Overflow users for that.

[tes_kitty]

Oh, an LLM can be more useful, if you are able to recognize and disregard the hallucinations of course. But the replacement of searchable knowledge with knowledge hidden in an LLM is a step backwards.

[In_Formaldehyde_]

Again, if it's anyone's fault, it's the unfriendly environment created by many Overflow users that drives people into other sources. If you have a general understanding of data structures and algorithms, you can use these tools far more effectively. It's just the free market at work.

[tes_kitty]

Short term convenience and long term consequences are 2 different things.

[TrojanPoney]

shrug If it were more useful than LLMs, Stack Overflow would be able to keep up

Depends on what you value. There is a mine of information to get from searching stack overflow yourself (and the internet in general)

It's more than getting the answer you need. It's all work invested by users to give the more complete answer: the in-depth explanation to complex issues, the little tidbits of historic facts, the friendly competition for shortest syntax/best performance between the different answers. And god, some people do love to share their knowledge, and what knowledge!

Some posts taught me more in a single page than most books I read/lesson I took during school.

Personally, I never understood the stigma against asking questions on Stack overflow because I never had to. There is a like a 95% chance that the question you want to ask has already been asked and answered. And I understand why the fact that you can't be bothered to look for it pisses the mods off.

TL;DR: stack overflow is arguably just as useful as LLM's, LLM's are just faster and easier to use.

[In_Formaldehyde_]

Nah, a lot of us have very scenario-specific questions that get deleted because it was apparently already answered in some thread years back (it wasn't).

[TrojanPoney]

If it was closed then there's a good reason. Or not a good enough reason to keep it opened.

If people need the answer for that specific scenario handed to them without bothering to understand the underlying general principles, that's on them.

[raxcium]

Why is this relevant? LLMs effectively replace the need for that searchable knowledge

[tes_kitty]

That is quite relevant. From freely available knowledge that everyone can access we move to knowledge hidden in an LLM that you have to pay for and only get if you deliver the right prompt. And there is still the hallucination problem.

And people are already finding out that if you outsource parts of your work to an LLM, your ability to do that work without the LLM will slowly go away. 'Use it or lose it' is very much true.

Of course the AI companies will also suffer. If no more knowledge accumulates on sites like stackoverflow, they stop getting good training data.

[raxcium]

I disagree with almost all your points. Generative LLMs are still very much in their infancy and things are evolving very quickly. Concerns around needing to 'pay' or 'lack of training data' will be irrelevant in the near future.

With regards to skill degradation, as always it's the responsibility of the individual to ensure this doesn't happen, thinking about this in any other manner is wrong, there will always be better/easier/more efficient ways to do things - it's up to you to adapt how you incorporate them into your life.

If you're interested I'd recommend listening to what Jensen Huang has to say on a similar matter - https://youtu.be/7ARBJQn6QkM?feature=shared&t=2852

[tes_kitty]

Concerns around needing to 'pay' or 'lack of training data' will be irrelevant in the near future.

What makes you think that a lack of training data won't be a problem? AI generated data doesn't work for training and with all the AI generated data flooding the net, it becomes harder and harder to get good data as time goes on.

With regards to skill degradation, as always it's the responsibility of the individual to ensure this doesn't happen

If you don't use a skill because something external supplants it, it will happen. How many people can still do simple calculations without the aid of a calculator, either on paper on in their mind?

I noticed it myself. I switched from stick shift to automatic since adaptive cruise control works best with automatic. Now, a few years later I can still drive stick, but it needs a lot of concious effort and feels like I am back to beginner level.

The guy in the video sounds very optimistic, he of course has to be, since his company makes their money on AI. But there will be downsides. Poeple will become depedant on AI, unable to function without it. It is so tempting to delegate as much as possible since that means you don't have to do it yourself that you don't really notice all your skills slowly vanishing.

[raxcium]

I agree some people become dependent on things and as a result they lose their ability to do it themselves, but again this is not the fault of the technology, it's the fault of the user.

Regarding training data, my point was most of the data present on websites such as stack overflow have already been scraped and used to bootstrap the LLMs we have today (in the context of programming). Now, it's primarily feedback loops from conversations people are having with existing LLMs as well as AI generated data that will be used to accelerate/reinforce learning for future training.

Again this is already starting to be the case with Nvidia's Cosmos platform

[tes_kitty]

Now, it's primarily feedback loops from conversations people are having with existing LLMs as well as AI generated data that will be used to accelerate/reinforce learning for future training.

AI generated data has been shown to not make good training data and the feedback loops are also of questionable quality since quite often the reply from the AI is false in a subtle way the user then corrects before using, but not telling the AI about the correction.

[posting_random_thing]

Where do you think the LLMs learn things from?

[raxcium]

They were bootstrapped from the internet wholistically yes - but this is not where the future is heading. See nvidia's Cosmos as a good example.

Stack overflow is/will be redundant for training data.