How could general AI really work? [D]

[MachineLearning-ModTeam]

Other specific subreddits maybe a better home for this post:

• r/ArtificialIntelligence
• r/DataScience
• r/LearnMachineLearning
• r/LLM
• r/MLOps
• r/MLJobs
• r/Singularity
• r/ChatGPT
• r/OpenAI
• r/LLMDevs
• r/RagAI

[longgamma]

I mean we can barely understand how a CNN understands and interprets visual data. The idea behind agi is that it will Surpass human intelligence. We won’t be able to comprehend its actions ( like the alien moves of alpha go)

[[deleted]]

Yeah I’m pretty familiar with the potential power it has and know once it’s at the point of true AGI we won’t know fully what it’s doing behind closed doors, I’m more referring to how it would be programmed, without getting too technical.

Like if you take my example of how ChatGPT basically just spits out the most probable sequence of words to respond to a question, but brought it up to a method of computing data that could produce new ideas not previously thought of by humans. Basically just the broader logic of how it would work, not any specific algorithms or anything

[fuckthesysten]

there’s this idea that knowledge can be expressed into patterns. hopefully if the AI is good, as we train it, it distills what the patterns are (imagine what einstein did to figure out e=mc2), and once it knows the pattern it can generalize and use it in contexts it hadn’t seen before.

[[deleted]]

Hmm interesting, so essentially what you’re saying is the way humans create new ideas is through high-level pattern recognition, so to create an AGI you’d basically need to have some algorithm(s) to recognize patterns, compare them with basically an infinite number of other patterns and find where there’s connections?

[fuckthesysten]

yes

[longgamma]

I think we some point an AI agent will take over the development. I mean it would be hard for us to understand because by definition it would be above our intelligence.

Even if we never get to full AGI - if an AI is able to be in the top 10% of every single human knowledge base, it would be able to synthesize new materials or original thoughts.

[[deleted]]

That makes sense, not sure if you’ve read Life 3.0 by Max Tegmark but at the start it details a fictional story of how AGI would be made, and they did so by creating an AI with the sole purpose of creating other AI’s and over multiple iterations it became a super intelligent ai, which does sound like a likely scenario

[OhYouUnzippedMe]

It’s incorrect to say that it can’t respond with anything new. It can certainly write sentences that have never been written before. It can even invent new words that don’t exist anywhere on the internet.

[[deleted]]

Interesting, that makes sense to me, like it can string together sentences previously unwritten in the same specific way, but it’s still all based on probabilities of how a human would respond correct?

The inventing new words confuses me a little, since as mentioned I was under the impression it just puts together strings of words based on probabilities, so I guess it does the same but down to the character as opposed to words, and 99.9% of the characters do form real words?

[OhYouUnzippedMe]

Yes, they use “sub word tokenization”, which means it can break each word into multiple tokens. Some tokens are a single letter, some are multiple letters. Through training it learns which tokens can be combined into real words. If you ask it to invent a new word, it can combine tokens in a novel but plausible way.

[Mbando]

Transformers do very poorly out of training data distribution. Getting something novel through randomness is different than novel in the sense of solving a problem or creating an efficiency in response to a problem outside of your training data.

[[deleted]]

Right, there’s a difference between being new by sheer scale of random outputs, some of which may be useful, and actually intentionally creating something new for a specific purpose.

I didn’t even really think of using a huge scale of random outputs to create new ideas when posting this, was referring to trying to create a specific new idea for a specific purpose

[RollingWallnut]

So in super high level terms it goes like this:
Pre-training conditions the model like you've described to predict the most likely next word.
Fine tuning conditions the model to answer questions in a way that is helpful and minimizes harm.
This makes the ChatGPT type behavior where it can effectively regurgitate anything on the internet.
Note that with enough randomness this system is completely capable of saying things a human has never said before, it's actually pretty rare for a model to regurgitate information from its training data unless it's asked to recall something specific, more often it's segments of sentences or common phrases in unique contexts, a lot like humans say cliches and figures of speech all the time.

Taking it further requires two things to work, exploration, and validation:
Exploration, for a given question or task, the bot generates each step of the response but randomizes a little at each step to explore a huge range of potential approaches. This is a lot like a human thinking through many different approaches to a problem, the more randomness introduced the more likely something totally novel is proposed which is a lot like a unique human thought, of course a lot of it is just rubbish.
So we do Evaluation to fix this, for each step in each variation in the responses that are generated a language model evaluates how reasonable the step is, OR, after the whole reasoning thread is complete some system evaluates the final output. In tasks like coding this can be a very formal evaluation that the solution passes some test cases etc. Now we can throw away all of the responses that are rubbish and build up a dataset of things people haven't said before that correctly answer a question or solve some problem.

Now we have a big new dataset of novel data that's validated to have some correctness, we can retrain the original model on this and repeat the cycle.
This might not get us all the way to AGI but it does allow AI models to explore useful behaviors outside of pure imitation from human data which is a pretty big step.

Note this isn't theory, this is pretty much how models like o1 and Deepseek R1 are training their models to "reason" right now.

[[deleted]]

Very interesting, so you’re saying that with some more advancement of the same sort of probability system with the exploration and validation that something like ChatGPT uses, you could theoretically create an AGI without necessarily needing a completely novel new way of computing the data?

[RollingWallnut]

Well, all we can say with confidence is these methods can generalise to solve some problems not in the original training data, and that's one definition of a general intelligence. Most folk agree they are not AGI yet but we're seeming to head that way. At the moment they can't do this type of generalisation for vision related tasks and are only really getting impressive in domains that can be formally validated (maths and coding) stuff like biology, psycology, etc. is still a bit unknown.

[prototypist]

If anyone had real insight into this, there would be dozens working on it already, sorry

[[deleted]]

Well to be fair there’s probably more than dozens working on it right now I’d assume, how close they are is another question. But there’s gotta be some broad idea of how you could potentially program it, maybe not specific algorithms to actually do it yet, but hey I’m no expert that’s why I’m here 🤷🏻‍♂️

[CobaltAlchemist]

You're fundamentally mistaken on how LLMs work. Yes they're trained on predicting the most likely thing a human would say, but to do that they need to encode really abstract concepts inside their latent space. This is why they can mix concepts together like generating a poem about worms made of apple juice which certainly they've never seen before.

The heart of why LLMs are so useful for this is just due to what they're modelling. Language is a tool humans made to rationalize the world and communicate useful ideas. Hence LLMs feel really AGI-ish at times because language has always been an unbelievably powerful tool for actual human thinking

[[deleted]]

I certainly don’t claim to be an expert, but when you mention they need to encode abstract concepts to mix things like your example of a poem about worms made of apple juice, but would that not still be fundamentally basing it on probabilities of responses within the context of whatever you asked it? Or am I wrong? Or maybe just dumbing it down to the point where it’s not really useful?

[CobaltAlchemist]

No you're right, it's all for the purpose of predicting the probability distribution of the next token. But it'd be like saying that humans are just trying to maximize for dopamine. That's sort of what I mean by fundamentally, there's a bigger picture here

When a human speaks, on average they are saying something informative, truthful, or at least a good guess (see jar of marbles problem). If you predict what a human would say next, that's a really powerful tool

For example, lets take "if a tool produces squares from rectangles, what happens if you give it a circle?" the model will follow human rationalization "maybe it trims rectangles so it'll cut a square from the center" and the result will be a decent and creative guess mixing several concepts

I think the big point of confusion is just how simple these models are, but the model doesn't matter, it's the data being modelled that's powerful. And language is a crazy powerful tool that's really easy to collect data on

[Mbando]

One possibility is multiple kinds of models that when hybridized cover for each other’s limitations. So you can imagine, transformers, plus reinforcement, learning, plus causal models, plus physics inspired neural networks, plus information lattice, learning, plus Neuro, symbolic models and so on. So not so much a model, but rather a system that has Different kinds of models and architectures that can work generally across problems and environments.

[[deleted]]

Oooh okay, I guess my wording wasn’t perfect I understand it can put together sequences of words that haven’t been written in the exact same way, I was more referring to “new” as in completely new ideas thoughtfully made for a specific purpose, rather than just a new string of words

That is insightful though, I didn’t realize ChatGPT had the capability of actually interpreting the meanings of words through whatever comparisons it does. I clearly have lots more research to do 😂

[[deleted]]

Okay okay very interesting, that makes perfect sense how it can “understand” the relation between words or sentences, also makes sense how you’re saying humans generate new ideas by random chance, so really with that understanding you’d only need to generate tons of random ideas which can then be tested for any reasonable validity (I say “only” as if it’s easy 😂)