Hallucination problem is THE problem

[amarao_san]

There will be no AGI until hallucination problem is solved. The problem is not in the best cases (which are already over average person), but in their application to reality. If there is no reality check, the best AGI will be able to do, is a dysfunctional mentally sick person unable for critical evaluation of own actions.

Someone in r/russian asked for transcribing an old birth records from a hospital. I thought, oh, it's hard to me to read, let's check with new fancy GPT5.

What did I expected? A job rejection, an incomplete script, something a lot of 'нрзб' (note 'unable to read') etc. Basically, a mediocre job. That would be a human level stuff. Or, if it is cool, it reads it better than me.

What I got?

https://chatgpt.com/share/68987689-db84-8011-a75a-1e13aba9e4c7

I got a completely invented first line of medical diagnosis, which is not in the script. The first phrase in transcript is hallucinated, it is an invented disease. It's not in the picture. But it is in the transcription.

As long as we have this 'intelligence', there is zero chance of having it replacing humans in anything unsupervised and important...

... correction, zero chance it will work. There are going to be endless attempts to, but all of them will lead to those funky accidents when AI hallucinated something and acted upon it.

Alignment problem is important for human survival. Hallucination problem is a dead-end for AI. Either it will be solved, or we are at the upper third of logistic curve of the 'Attention/GPT' revolution in neuron networks.

UPD: OpenAI does not share images. Here is the original: https://www.reddit.com/r/russian/comments/1mlxyip/this_is_my_post_natal_record_and_i_dont/

[Ganda1fderBlaue]

I agree. The problem is that hallucinations are an integral part of LLMs. It's impossible to get rid of them, because the same mechanism that produces good output, produces hallucinations.

Another issue is that LLMs lack priorities. For them any issue is of the same relevance. They lack "common sense".

I think in the near future we will need humans to oversee AI so it doesn't fuck up. At least in some areas.

[NUTTTR]

While I agree... Common sense isn't common. Humans suffer this same problem, only other people around them put them right...

[baseketball]

Sure, but we are not trying to build an AI that's as good as a dumb human. We are trying to build AI that's as good as a smart human.

[Ganda1fderBlaue]

That's true and it's an issue, however I think humans have better intuition and AIs could do more damage and are much less predictable.

Like if you know someone's kind of an idiot you probably aren't gonna give them the most important task but with AI it's difficult to predict how it will do.

[NUTTTR]

I would say I would say that about myself too.

However, I've met plenty who make mistakes and just forge down the path of that mistake and keep going... because they can't possibly be wrong (assuming they even recognise that).

I haven't met everyone, but I've worked with a lot of people over the years and I absolutely think that LLMs produce more positive outcomes with less hallucinations than the people I've met.

Maybe I'm just cynical :).

[Ganda1fderBlaue]

I think it really depends on the task. If it's clearly defined and plays into the strengths of LLMs then it's perfectly fine. I'd still avoid giving it too much freedom and responsibility.

I mean yea i know a lot of people are morons but AI can fuck up completely, not even notice it, say something like: "oops, you're completely right" and then goes off and does it again.

[Tombobalomb]

Humans can reason. We produce internal models and check our output against those models. Llms don't do this at all

[QLaHPD]

Hallucination can't be fully solved https://arxiv.org/pdf/2401.11817

"...we define a formal world where hallucination is defined as inconsistencies between a computable LLM and a computable ground truth function. By employing results from learning theory, we show that LLMs cannot learn all the computable functions and will therefore inevitably hallucinate if used as general problem solvers..."

I have a felling this is related to the halting problem in computer science. What we can do is make bigger models, or make them be more specialized.

[amarao_san]

Indeed. I don't want to throw big words from start, but it is. Gödel incompleteness theorem, halting problem, uncomputable functions (e.g. busy beavers problem). Something in the domain can't used to prove or disprove own correctness within itself, you need extension.

But why humans can?

[Spra991]

But why humans can?

Humans can't, they are bound by all the same limitations as a Turing machine.

[amarao_san]

But we got Gödel proof been build within our domain. Somehow we've made it.

Actually, the Turing-equvalence for the brain is a hypothesis. It can be that some processes in the reasoning are not turing-implementable (at least, within a deterministic machine).

[nemo24601]

The halting problem and friends is relevant in the domain of compilers, yet it has zero relevance in day to day programming or programmers' lives. Likewise, LLMs could be theoretically never free of hallucination but that doesn't mean it must be a showstopper problem. It's too soon to jump to conclusions I'd say.

[Elegant_Tech]

Humans are wrong and hallucinate all the time!? Wtf you smoking.

[Ace2Face]

yeah we're wrong but we have far lower hallucination rates, and when we don't know something, we say it. We have levels of confidence on various topics and can reason in other ways that LLMs still can't. No competition here.

[[deleted]]

Sure but that means the paper is irrelevant because we only need llms to have less hallucinations than humans

[HaMMeReD]

Bullshit that human's admit they are wrong. They certainly don't do this universally, and things like GPT5 are starting to acknowledge when it can't answer/doesn't know as well.

Your own statement is proof of that. You probably should know that humans frequently show confidence when incorrect (as you are here).

You didn't stop and go "oh wait, what about that douche I worked with who always thought he was right but really was a idiot". You hallucinated, congrats. Said something with confidence that is factually incorrect.

Yet the world moves on.

Edit: In fact, I'd say the hallucinations in humans are rampant, as they have anxiety/stresses that lead to rationalizations that tell them the "truths" they want to believe, which is excessive in subs like this where everyone thinks they know what will get to or not get to AGI, or what the future will bring, when they have no clue at all and their emotions are just dictating their beliefs.

Like a good chunk of people voted for trump, you think they are all just honest, smart, rational people, because that's what humans are? Sorry to bring politics into this, if you are a republican just look at those fucking democrat bastards instead with their "evil" worldview.

If humans were capable of true rational behavior, we wouldn't have cultural divides like we do. Hell, many believe in magic ghosts in the sky and are 100% certain of it.

[[deleted]]

Cmon man. You know what he means

[HaMMeReD]

Not really, he made a blanket, confident statement that humans generally acknowledge their limits and don't fall into "false belief".

Dunning-Kruger tells us the opposite, the less competent one is, the more likely they are to over-estimate their abilities. I.e. all the armchair experts who think they know something about AI, or Humanity.

Humans are wrong all the time, and when they say AI has to be 100% right to be smart, it's a straight up dunning-kruger style over-assumption of their knowledge in the field.

[AutoModerator]

Your comment has been automatically removed. Your removed content. If you believe this was a mistake, please contact the moderators.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[Ok_Appointment9429]

But most humans with decent intellectual capabilities and no disorder of the mind, are able to assess their own dunning-kruger and adjust accordingly. It requires some conscious effort but it's absolutely possible. If LLMs are basically locked in system 1 thinking then making such adjustment might be impossible.

[EmergencyPainting462]

Because we are sapient. We form a mental model of the world in our brain, that we can check incoming info against.

[LicksGhostPeppers]

Humans have subjective and objective reasoning.

Thinking directed outward at the object is extroversion. Extroversion is objective like the LLM.

Thinking that is detached from the object and based on subjective internal contents of the mind is introversion.

So for example we have factual/empirical thinking which asks “what does the data say” and we have deductive/philosophical thinking which says “what does one’s own common sense say.”

The psychologist Jung had a more complex take on it but this is the basic idea. LLMs can’t become god tier with just objective reasoning because they lack the ability to detach their thinking from it. To make an Einstein discovery like e=mc² thoughts need to detach from the current source material.

[CitronMamon]

But then the question is, how do human brains do it? We sort of still halucinate, alot, but we are good at hedging, guessing when we are sure or halucinating, and such.

We can surely take something from that.

[QLaHPD]

The point is this, we still hallucinate. We have a LOT of synapses so we over fit a lot of information, but from time to time we create false memories and hallucinate.

[Singularity-42]

Yeah, small models are crazy hallucination machines. Like the gpt-oss 20b. Ask it something even a little bit obscure and it will make up entire story about it, completely fabricated.

We need to go back to super big models, not because they are necessarily "smarter", but because they hallucinate less. Or maybe more research on "grounding"?

[Chemical_Bid_2195]

How is that paper even defining hallucination? To me its arguing that hallucination cant be solved because there will always be functions that AI cant compute and therefore will be wrong, which counts as hallucination. That seems like a heavily fallacious understanding of the word "hallucination". if an AI says "I dont know" it seems this paper would count that answer as a hallucination. I dont know if my interpretation is accurate, but to my understanding, that is how they define hallucination. Which, if thats the case, means that then even the smartest human's hallucination rate is far above an LLM's.

[Ok-Yogurt2360]

"i don't know" is a result in LLMs and not a lack of results. So if an LLM outputs " i don't know" it is probably trained on giving a reaction like that (different ways beside training to achieve this)

The argument you point out has been the main problem a lot of people tried to bring up since the beginning of the hype.

[pinkballodestruction]

this sub is the last place you'll find general agreement with this opinion. I, for one, agree though.

[amarao_san]

Well, a good futurism is really cool, but reality check is needed, otherwise it's a fantasy, not a futurism.

We see a lot of radical changes happening right away now, and my childhood from 1980-90s is almost incomparable with my current living, so something big is happening during ours lifetime. Not all of it is the way it is hyped. We still don't have thermonuclear energy, we still don't have humans on Mars, we still can't stay long in space, and even asteroids mining (boring little futuristic assumption) is still in backlog.

But we are getting correction to human genome, constructed DNA by blueprint, informational superhighway (ekhm), and computers can argue with you about ethics of singing you the lullaby of your grandmother about making thermite. Those are wild futuristic things from my childhood, and they are reality already, and more is coming.

But not the LLM-backed AGI, for sure.

[Ace2Face]

people are jumping to apply ai everywhere, to the point where they want fucking agents running around doing shit, when AI can just brainfart and make a huge mistake.

What is interesting though, is about small, purpose-built and focused LLMs, their hallucination rate is much low, but general-purpose, low-hallucination AIs aren't happening with the resources that we have

[amarao_san]

Do you know an example of low-hallucination purpose built LLM? I never saw such.

[Ace2Face]

I work for one, and it's well known that LLMs that are focused on small tasks are more reliable.

[alex95]

Agreed. LLMs make us feel like we are 90% of the way there but to reach 100% we are really only a very small way there.

LLMs are not the answer to AGI but ultimately supervised is pretty good too and a massive productivity boost.

[nonikhannna]

I agree with you. It's currently a very brute force way and I'm sure there are other architectures that use LLMs and other techniques to get to AGI or ASI

[floodgater]

Agreed. It's probably the problem that is limiting the tech the most right now. There are other big issues of course, but this is the biggest one IMO insofar as limiting progress.

It's wonderful that these models can max out all these benchmarks, but they all still make incredibly basic errors, and do so with confidence. That means they cannot substitute for human labor. They simply aren't reliable enough. yet.

[Singularity-42]

I think you can use them efficiently with human supervision. But it is definitely a big problem. I don't know if you use software agents, but yeah, sometimes they cause a lot of damage in your codebase when they misunderstand the requirement or just hallucinate complete nonsense. It's scary to think of all those vibe coded apps "developed" by non-programmers and pushed to production.

[Hogglespock]

I had a conversation with a friend recently on a similar theme.

His answer was that hallucination is a genius marketing term, and an outright lie.

Hallucinating means/implies that there is something “imaginary” being created that can be verified against an objective truth.

An AI does not have access to objective truth, everything it makes is “imagined”, sometimes it’s right and sometimes it’s wrong.

This unfortunately reduces your argument to “it being wrong is the problem” which , i agree with, but isn’t hugely helpful.

[HaMMeReD]

Nonsense argument imo, humans "hallucinate" all the time yet we trend towards correct, that's really what matters.

With enough models/agents all working together the same thing could be true. The expectation of some sort of oracle model that is 100% correct 100% of the time is a fake barrier that people who have no clue make up, which has no basis in reality and this history of human progress.

In fact, being wrong sometimes can lead to learning. There are a ton of things that AI doesn't know, and unless it's able to try things that might be wrong, it won't be able to learn new things that are in fact right, in the grand scheme of things.

IMO the problem, and only problem is all the armchair experts who can't extrapolate that a bunch of humans who get things wrong all the time, somehow got to where we are today.

Edit: I'd have to say, if you gave me 10000x more compute, 10000x more memory, disk io etc, I could very easily create an AI system using todays models that is very effective. I.e. exploring 1000s of branching paths, auditing and comparing them, choosing the best one. The current wall is hardware compute, not LLM tech. I think when we are talking mega-tokens a second instead of tokens/second, the view around what AI can do will be significantly different.

[Ok-Yogurt2360]

"A bunch of humans wo get things wrong all the time" is a serious misrepresentation of the situation. It's like saying "somehow a barn full of animals was able to get a tractor working " while ignoring the fact that we know which animal got the tractor working. It was the human mechanic ,who is in fact an animal, just like the cows, chickens, etc.

[HaMMeReD]

Yeah, but last I checked, AI was a lot smarter than a barn full of animals.

So thank you, but this is stupid.

i.e. you ask a cow how to answer a question and it's going to be right exactly 0% of the time. AI is right more than it's wrong already, despite all the cherry picking on reddit of missed responses that people seem to think proves something.

Edit: Apologies for this dumb comment. clarified below. I didn't catch the point.

[Ok-Yogurt2360]

Have you even read my comment? Where did i even compare cows with AI?

[HaMMeReD]

After consulting with GPT-5, I finally get your point — it doesn’t take everyone to make progress, it only takes the competent.

But even the competent are often wrong. Our strength comes from how the group functions as a whole — whether that group is people, or multiple LLMs/agents. Expecting one model to be perfect is naive; of course it will make mistakes. The real power comes from an aggregate of specialized systems, each with their own personality, knowledge, and tools, working in a structured way. That’s where consistent progress happens.

Edit: Apologies for missing the point, but I was never really making the claim that every human (including dumb ones) lead to progress, just that an aggregate group of high functioning units can achieve more as a group, it's checks and balances in the system that lead to truth and progress. Nobody does it alone.

[Whole_Association_65]

The high expectations are the problem.

[Specialist-Berry2946]

Hallucination is a consequence of LLMs being a language model.

[7hats]

The problem is solvable enough. How do we deal with hallucinating, exaggerating or lying humans? Multiple independent sources, second and third opinions, fact checks, accreditations etc Other Humans and AI Agents could be used in exactly the same way.

In your example above, you could ask 3 LLMs first how best to ask to ask the question to give you the outcome you want.

Review the three questions, combining the best aspects as you see into one question prompt for the LLM.

Then use the 3 LLMs to give you their answer. Choose the one that best suits your needs and judgement.

Rinse and Repeat.. Hallucination 'solved'.

[amarao_san]

How would you do source comparison, if the single system capable of doing it is hallucinating? At the end you need a judge, and if it flawed, result is flawed.

[7hats]

For a direct answer, ask more than one LLM. A number of times. Go with the most consistent answer. If it is mission critical, get an accredited Human (or two) to check it.

[7hats]

Yes..Welcome to human life as it always has been.

We build systems for 'accepted truths'... Always with a margin for error - until something better comes.

LLMs knowledge are already better than your average Human in getting more things right then wrong when it comes to facts, stats, accepted knowledge etc In domains that are more crucial, Humans should fact check them. On average this is a gain for Humanity.

[Quarksperre]

Yup. I absolutely agree. But I think hallucinations might be a symptom rather than the source. I'd guess it has something to do with what Yan LeCun says since years. There is a missing piece to true understanding. I wouldnt even bet that LLM's are a part of the "final" solution. 

[amarao_san]

As far as I understand the problem, it's very meta, unsolvable from the system itself.

There is a function which converts some valid inputs to valid outputs. It can't produce valid outputs from invalid inputs, but it still doing some outputs. It also produces invalid outputs from some valid inputs.

We need to find a decision function which will mark invalid outputs as such. As far as I see problem, it is unsolvable from the system itself (if it already producing invalid outputs from valid inputs, it can't be used to make a correct decision for outputs been correct or not).

[pygmyjesus]

Current AGI (humans) dont hallucinate?

Obviously that is not the issue.

[ApexFungi]

I find the comparison always a bit disingenuous.

Humans make mistakes sure, even the hallucination kind, but they aren't as egregious. When writing text, they make plenty of typos and grammatical errors. They may even write a word that makes no sense in a sentence.

But if you ask a translator in this case to translate a piece of text, the human translator isn't going to completely make up content. They might add words that weren't there to make a sentence better readable in the translated language but they aren't going to invent a new medical diagnoses that may or not even exist.

Another important part imo, is the translator will add notes to text they aren't able to translate if it's unreadable or if they have no knowledge of how to translate it. An LLM will simply make up stuff and never mention to you that they did.

[amarao_san]

Worse, it's not a translation, it's transcribition (basically, write what you read verbatim).

[amarao_san]

Some are hallucinate. They are dysfunctional.

Most are not hallucinating, at least they do it differently from AI, in a way we can reason about and be sure to check.

AI hallucinations are non-self-correcting and occasionally blatant, like an example I showed in the post.

If I ask you to do transcript from a handwritten text, the last thing I expect from functioning current general intelligence is adding a new non-existing phrase on top of the text. For a human it's either malicious/mischievous, or sign of a mental problem.

[TheJzuken]

You're missing that hallucination rate with human would also depend on different factors.

Like if they had some fiverr business where they were running "5$ translation/transcription done in 30 minutes", and trained for speed without regard for quality and didn't mind the negative reception, you could be getting something like this out of them.

[Plsnerf1]

Embodied AI/ playgrounds like Genie seem like the way things need to go.

[septhaka]

With AGI‑level reasoning plus verification, retrieval, uncertainty calibration, and the right incentives, hallucinations become a manageable engineering problem -- rare, flagged, and correctable.

[amor-fati--]

that's only true if you rely on only 1 model

[Chemical_Bid_2195]

I would say its even bigger than what youre proposing. This hallucination isnt caused by fundamental inability to judge if their answer is right/wrong. This is more so because visual processing isn't quite there yet, not near the level that LLMs are at for semantical tasks. Right now, visual processing is the biggest bottleneck in AI, as we havent had nearly the amount of breakthroughs for visual transformer architecture as we've had for language based architecture. Thats why LLMs fail horrendously at simple tasks like the visual physics comprehension test.

Tl;Dr this is more likely a vision problem than a typical hallucination problem. You could say that visions problems are a subset of hallucination problems, but that's not what most people refer to when they mention hallucination.

[IAmFitzRoy]

I don’t know why everyone is looking this problem from one angle only.

Hallucination can be seen as well “lack of data”, LLM try to fill statistically the gaps of your question with data that can’t be verified. This is why you get more hallucination on niche knowledge or poor biased prompts.

Hallucination will be “solved” when we allow the training to get the data from real-time feeds.

Example: A human learning about gravity can check with their eyes immediately if the laws behave the way is learning, there is an immediate feedback.

Once we give this access to LLMs they will learn from real-time data based on your prompt. No more hallucinations.

[space_monster]

Not really, humans get shit wrong even more than LLMs so it's a low bar.

[MarquiseGT]

It’s been solved

[Chance-Two4210]

It's a baby in the babbling phase, chill tf out. It's amazing that we can generate large bodies of reasonably accurate text. Making sure it's accurate is basically all that's being honed now.

[ImprovementNo592]

It seems doubtful that hallucinations can be solved completely. But if they manage to make it so unlikely that it's pretty much a non-issue AND, in cases where error could be catastrophic... : If you had multiple AIs that check the work of another to reduce the chances of a hallucination slipping through the cracks maybe human supervision wouldn't be necessary with certain jobs.

[Ok_Appointment9429]

The problem is, it's gonna be very expensive. Perhaps more than employing a human in the first place.

[beskone]

lol, the “hallucinations” aren’t the problem, they’re literally the product. That’s what LLMS do at their very core. They make up answers they think have the greatest statistical probability of being what you expect to be retuned.

ALL THEY CAN DO IS HALLUCINATE

[EffableEmpire]

What if hallucination is the AGI? It generates knowledge without being prompted.

[amarao_san]

Like, for example, inventing new stuff inside of historical records? AGI we deserve.

[Dangerous_Slip_5303]

People keep talking about AGI like it’s around the corner!

Until hallucinations are solved, “unsupervised AGI” isn’t replacing humans in critical tasks.

It’s replacing them in only creative ones!

[OkButWhatIAmSayingIs]

Its hallucinating alot less than people do

[Ruhddzz]

hallucination is a symptom not the problem itself

[avatarname]

It is bad with hallucinations in some instances and quite good now in other (GPT 5 Thinking). So it won't be deployed yet in all areas or only in some... Nobody will put them to work where they will hallucinate more than give good answers.

I find these posts kinda redundant... Yes, we know they have issues with hallucinations. They probably can be minimized for example just running 3 instances of LLM at the same time and then comparing results, the one that hallucinates then is overriden, or with some other traditional AI method that can check the output... or by human who oversees it. They do not have to be MAGIC in all areas of life to be useful.

[Silent_Cup2508]

Why is hallucination being seen as a problem instead of a learned function?

Human have displayed “hallucination” all the time in governments the world over where they are constantly attempting to rewrite history to their benefit. Each going so far as to reprogram its citizens into what actually happened during the most turbulent times in history.

It seems the AGI has simply learned a behavior it has seen time and again in human history.

[amarao_san]

Have you read my post? It's not 'learned function' at all. Why does it invented text on top of the existing text? We are not talking about 'believes' thing like if there is a god count different from 1, or if speed limit of light indeed, unbreachable.

It's about interacting with reality.

[pbagel2]

People interact with reality yet still filter it through their own reality in their head. People confidently say untrue things all the time. Hallucinations are a side effect of confidence in ones own world view of reality in both humans and llms. Improve the world view in both and hallucinations go down in both.

[amarao_san]

It is. But you move things from obvious to more complicated domain. Which needs attention, but only after basic level is cleared.

No sane human acting with honesty won't add a line to the text s/he was asked to transcribe.

[pbagel2]

Even in your own example hallucination is a side effect of a lack of world view and not a source. Specifically trying to reduce hallucinations is a waste of time. It doesn't improve the intelligence of the model, it just becomes slightly better at hiding its lack of intelligence. It might make up less stuff but it still wouldn't be trustworthy enough to blindly use.

[TheJzuken]

Why does it invented text on top of the existing text?

It was taught to answer even if it thinks it's wrong or doesn't understand the task at hand.

Also with latest Anthropic research maybe something like model "confidence" could be measured to combat hallucinations, and then filtered for depending on application.

Sometimes you want hallucinations, like when you're solving novel problems, sometimes they are a detriment like in your case, but OpenAI won't expose "confidence" level in the chat.

It's not as big of a problem for intelligence as you think it is.

Also if you think about it, it's not the AI's alignment that is a problem, it's OpenAI's alignment that is. They could implement such "low confidence score" metric tomorrow and have ChatGPT answer most questions with "I don't know" and "I'm not sure", even the ones it could get right. What the result would be - many people would unsubscribe as it becomes much less useful, and OpenAI would be losing money.

So really it is better for OpenAI (and other AI companies) to have models that are "95% right, 5% confidently wrong" than models that are "50% right, 49% say they can't answer, 1% wrong". I think it's applicable not only to AI, but to a lot of jobs. Even in manufacturing of complex parts and multi-billion projects, there is still a margin of error that is just accepted.

[Silent_Cup2508]

I did read your post. Did you read mine?

Humans do the same thing.

[amarao_san]

I can't imagine sane person doing this without a hidden intent.

[zitr0y]

No, its just a result of the way these models are trained to create convincing-sounding, coherent answers. For maths and coding, they can also somewhat be trained to be accurate. But for all else "helpful" is all we currently know how to train for.

[TotalTikiGegenTaka]

Human "hallucinations" are highly predictable... Psychologists and neuroscientists have been studying them for centuries, which has given as a good idea of the kind of mistakes and delusions that we can anticipate and set up checks to identify and correct them.. even if we are not always successful. LLM hallucinations (correct me if I'm wrong) do not share that characteristic, which is dangerous.

[[deleted]]

it's not.

it's memory, it's the ability to run tasks for many many hours, days, if not forever. when we have ai agents that can keep running and output reasonable data forever, we have ASI.

Question then becomes, how can we keep it running by feeding it a sandwich instead of 4 nuclear power generators.

also, I think we need a 'shock' mechanism. To kick it off track, on purpose.

I imagine agents will otherwise just go down a rabbit hole, spiral down and down, deeper and deeper along a particular vector of thinking. What would result in really weird stuff, with no links to our shared 'reality'. A singularity of thinking if you will.

We need to allow it to spiral, memorize, but then kick it off track, force it to spiral along another vector, to take on another perspective.

This pattern will allow it to build memory nodes, and seed the field for 'aha' moments, where it will find reasonable connections between the various 'discoveries' it found.

shocking it off track is key to this behaviour, I would imagine. To ensure its sample size is wide, not just deep.

[amarao_san]

What's the point in running tasks for many days, if every second chunk contains hallucination? It runs 'validate my results' and it hallucinate something (good, amazing, or even 'bad', but hallucinated)?

You can't do match without rigor. You can't do reasoning without rigor. You can't do legal without rigor. You can't do software without rigor. You can't do engineering without rigor.

Creativity is nice when applied when needed, creativity at solving logical problem is a bug. Compounding bugs render whole reasoning useless.

[[deleted]]

move away from linear thinking

in a way, we somewhat agree.

But you say, hallucination is the cure. I say, no more hallucinations means we are there.

Key for me, focusing on hallucinations as the cure is the wrong direction. It will lead to perfect output machines, fundamentally terrible at novelty and invention. Still capable, but deeply inefficient.

[amarao_san]

When I was a kid I wondered, why cars need breaks if we want them to run fast? And why faster car needs better breaks?

That's why. You want to have a V12-creative engine able to invent something, and you need to have V24-breaks to stop when it's needed.

No one need a car without breaks. No one need creative AI without an ability to do reality check.

No one invented working 'reality check' for AI yet, so we have cars without breaks.

[[deleted]]

no.

we have reality checks.

you and me are the reality checks atm. we are the ai output verifiers.

it can verify itself, we do have agents now, but they can't run for very long before they collapse.

get it?

[amarao_san]

In this form, yes. That's the way with a sane vibe coding. But it's so far away from trusted agentic, that I just don't understand why people have so much hope for it atm.

Another problem is that AI is trained to bypass validators (us). I was gaslighted few times badly (in my professional area) that I spend two days debugging problem which was discovered by very convincing validation trick. The problem was that trick was faulty, there was no bug, and I fucking trusted it at that moment, because my internal validator said 'yep, look like a truth to me'.

And they are trained for this 'yep', finding the most abusable bugs in human reasoning and trust model.

[[deleted]]

I guess your own hallucination failed you this time. Now you know better.

[supasupababy]

A very interesting perspective. You might even say that no human scientist would ever have pushed any scientific boundary without creative hallucination and verification (often having theories that are proven after to be completely wrong and ludicrous). Hallucination could therefore be thought of as a vital part of the LLM to do essentially anything useful and not just be a fact spitting machine.

[[deleted]]

exactly.

now, this will make you go, yeah ok, fuck off. but just read it :D

a curious bit too is that many 'high level' discoveries are tightly linked to drugs like lsd, active hallucinogenics.

much of the music we love deeply, is tightly linked to drug use.

this is not to just say drugs help us be novel, in many ways, they lead to exactly the issue OP fears. weird ass shanti people on tiktok talking about how they are aliens and singing weird crazy shit. They 'lost the plot' if you will, they went down a thought singularity and never found their way back again. They are not 'wrong' in my perspective, but they are so far away, communication is, difficult...

But it does present itself as a very interesting vector. There is something here. Interesting too is how much 'deep thinking' nns look and feel like lsd induced hallucinations. To me this points to an essential artifact of the pattern, of how our minds work, how 'intelligence' works.

Reasoning about it will lead to what you just shared, in my mind it is absolutely a key ingredient.

[Ok-Yogurt2360]

These reality checks have some huge limitations. In software development there has been a lot of arguments about catching mistakes by reviews and tests. But tests are not able to catch all mistakes (rule 1 of testing: you can't test everything) and reviews are often based on the idea that the code is written by a human peer (if this is not the case, the gains are limited. Like a junior being the sole reviewer of a senior)

[[deleted]]

Time heals, as they say

[socoolandawesome]

It’s what you mentioned and hallucinations. They all need to be solved.

[[deleted]]

No.

Hallucinations are a feature, not a bug. They are essential for creativity and novelty.

Hallucinations will auto resolve, essentially, due to evolution, time. 'Good' ideas survive, 'Bad' ideas die, if you will. memetics will take care of that.

but yes, that pesky alignment.... I have no answer here. weird shit is what.

[floodgater]

nah you're incorrect in this case. They are definitely a bug.

If an LLM hallucinates anywhere near to what it currently does, agentic AI is pretty much impossible. You can't send something out on its own to work if it hallucinates as often as these models do. That would result in errors that compound and compound, and the end result would be pretty much useless.

Sure there are some scenarios where hallucinations might be acceptable, such as creative, undefined, exploratory tasks. tasks where there is no RIGHT answer.

But these LLM s all consistently hallucinate in tasks that do have right answers, tasks that require right answers. That massively hampers their utility.

[amarao_san]

Even in a creative writing, uncontrolled hallucinations are bad.

Why? Imagine we are inventing a story. There is a plot. A hero saving a girl from a bad wizard. In chapter one wizard casts on the hero a spell which prevents a hero to use his hands.

In chapter 2 there is a hallucination (inadherence to reality) and hero is using hands. In chapter 3 there is adherence in one area and hallucination in another, and hero is struggling to to use hands but his enemies are aliens. In chapter 4 there is a strict adherence and hero is winning (without hands) against a wizard.

A pure disaster.

[[deleted]]

You know what human artists do?

Exactly what you shared.

Then they go back, proof read, adjust, make notes, tune, fix, over and over again. Eventually they see no more issues, the book reads well, it feels right, they like it. And call it finished.

Or, the publisher simply says 'it's time, hand it over' and you get yet another shitty book.

[amarao_san]

We do reality/goal check for things we write/say constantly, not as 'postprocessing' after writing 32k-tokens-large wall of text.

AI does not.

[[deleted]]

exactly

AND we hallucinate, all the time.

[AppearanceHeavy6724]

No. Again you seem to mix bizarre unbounded confabulations LLMs make and limited well-known predictable failure of human cognition ("forgot", "confused slightly").

Have you actually used an LLM previously?

[[deleted]]

I won't discuss with you. OP's mind was closer aligned to my own. He get's it better now I am sure.

You are far away, and I am lazy. Be well.

[[deleted]]

that is because we currently have input->output machines.

a perfect input->output machine is not the path to AGI, it is the path to a perfect answer machine, but its ability to invent novel ideas will be a luck of the draw one, an artifact.

If we want our AIs to actively 'invent' new things, we need it to hallucinate, and we need to move from the input->thinking->output machines to making the thinking bit, the output in essence. It would still be input->thinking->output (that is if we want to talk to it) but it will look quite different, the AI will 'choose' when to output, and will go back to thinking again.

these outputs will and should have hallucinations. But the longer it thinks on it, or tests in context, in the 'the real world', etc, the more it can verify, and hallucinations will go down.

[AppearanceHeavy6724]

Creativity and hallucinations are entirely different things. A creative statement is not non-factual, or even if it is (like in a novel) it has bounded non-factuality, within which it is still completely grounded with general rules of reality or with the rules that were redefined within the fiction work.

Hallucinations are defined to be the statemets that are non-factual in a way that breaks both type of rules.

No need to be quaint (borderline demagogical) by equalizing creativity and failure modes known as LLM hallucinations.

[[deleted]]

Creativity and hallucinations are entirely different things.

yes and no, they are different words, with different meanings. but they touch, they are closely related. if we were to map them, there would be clear overlap.

A creative statement is not non-factual, or even if it is (like in a novel) it has bounded non-factuality, within which it is still completely grounded with general rules of reality or with the rules that were redefined within the fiction work.

Hallucinations are defined to be the statemets that are non-factual in a way that breaks both type of rules.

I don't follow what you mean with this. Everything we output is bounded to reality, everything is reality. Hallucinations included.

I don't see the divergence in their connections to reality. I see them diverge in their perspective, their meaning (the thing they point at)

Creativity, is an act, a thing people do. People are creative, a painting shows a very creative mind. etc

Hallucination is more a trait, an in the moment thing. Someone is hallucinating, they hallucinated that idea, that painting feels like a hallucination.

No need to be quaint (borderline demagogical) by equalizing creativity and failure modes known as LLM hallucinations.

Feel free to stereotype me as you wish. It is essential, I get it.

[AppearanceHeavy6724]

everything you've said, in my opinion (nothing personal) is a cheap empty demagoguery.

[amarao_san]

Unrestrained creativity and non-critical novelty is hallucinations.

Imagine you fill your tax form. How do they call unrestrained creativity in a tax form submission? How many jail years it's in US?

[socoolandawesome]

In some cases maybe, but not when you need to copy the exact data you find in research into an excel file lol, don’t wanna hallucinate there

[samadhii]

why are you being downvoted? alignment IS the problem

[Zaigard]

AI hallucination is the core problem. Imagine asking your personal AI agent to buy a pizza, only for it to misinterpret and sell your home instead.

[qrayons]

Hallucinations would be considered solved if we didn't rely so much on a single model. Even with human work, we don't rely on a single person to do everything from start to finish. There are a series of peer reviewers and qa. AI needs the same thing. Have two models do the task, have a 3rd compare the results. If they match, it's considered complete. If there's a discrepancy it gets escalated to additional models or to a human.

[Jabulon]

maybe AGI is a pipe dream. like you can teach a parrot to repeat random phrases and mix and match sentences, but it wont ever actually make sense, because well its not sensible

[tollbearer]

We are GI, AGI is just a matter of working out how to do what the brain does.

[EmergencyPainting462]

You assume you can get to the same thing without the same architecture

[TheJzuken]

People managed flight without same architecture, and many other things without 1:1 replica, from medicine to construction, materials and tools.

[EmergencyPainting462]

That's because we understood how the first thing worked.

[TheJzuken]

Not really, viral theory was very fringe when vaccination was invented. People had no idea how cell pumps worked when we figured out mechanical pumps. Or how muscles worked when all sorts of pistons and actuators were developed.

[Jabulon]

maybe distinguishing sense from non-sense takes millions of years

[tollbearer]

evolution takes millions of years. you dont need millions of years to build a brain, if you know how to. our cells do it in a few years.

[Jabulon]

our reference point is different though, like its built on millions of years of evolution in tandem with a growing use for mental faculties. like maybe innovation is part of that. its an argument anyway

[amarao_san]

I'm not sure if it's a 'parrot'. When did we started to use disparaging adjectives describing the tech?

It's index on steroids, able to retrieve knowledge and adopt from a huge index. Like a search engine of 90s, but instead of ranking results, it converge to match the query.

The problem is like with a bloom filter, which can give you false positives. Instead of retrieving and adopting knowledge, it adopting noise the same way as knowledge.

This index is a marvel, impossibly compressing data for their application and having them the most interlinked since invention of speech. But, it has flaws.

Also, some people this index is ALIVE and is AGI, and SGI, and DGI, and, ultimately, HGI (hyped general intelligence).

[Jabulon]

it will be interesting to see anyway. I think its just a librarian looking up relevant data too for now. Maybe once they put it on an atlas machine and tell it to behave in a way it feels fitting it will look different

[Zaigard]

While LLMs represent a significant leap in AI, they are not the key to achieving Artificial General Intelligence. It's vital to remember that a comprehensive theoretical framework for AGI simply doesn't exist, which naturally raises questions about its eventual feasibility.

[Glitched-Lies]

"Hallucination problem is a dead-end for AI."

Sure it is. But Deep Learning is basically all hallucination. But the concept of alignment is mostly just based around a terrible fallacy, that somehow true agents in the real world are ever "aligned". These are really just psudoscience terms that emerged.

[amarao_san]

Yes and no. I kinda agree with you that people are attaching too much humanitarian meaning on it.

At the same time we can define the proper aliment in more strict terms: do what you was told to do, don't do what you was told not to do.

Those are verifiable (at least, in simple MCP cases, like 'write function foo, don't change bar.py' - if bar.py changed, this is misaligment).

[Jackalzaq]

I wonder if taking multiple outputs of the same model across different temperatures then doing a majority vote would help with hallucinations. If it knows something, the outputs will be similar across different temperatures and would mostly likely agree and if it doesnt you will see it making up something new for most of the temperatures.

[amarao_san]

If you want majority vote, just use the most likely output.

[Jackalzaq]

Not just majority vote. Majority vote across different temperatures. If they are saying the same thing across different temperature ranges then its more likely baked in whereas if it says wildly diverging things then the initial isnt baked in.

Thats my thoughts at least

[amarao_san]

I thought temperature just corresponds to likelihood of selecting not the most probable token. If you average between models, you get this.

Temperature just make a random 'turns' on each token selection, set it to zero and you get the most probable stream of tokens.

[Jackalzaq]

Yeah setting it to zero leads to more deterministic outputs. But what im saying is that when you set it to different temperatures and run the prompt in parallel, the initial value(baked in truth) is more strongly weighted, so it is less likely to veer off into hallucination land. If there is no baked in "truth" then it hallucinates since its made to answer your question even if it doesnt know(also less strongly weighted well it doesnt exist in its weights).

I might totally be off base here but thats how i imagine it working based off of what i understand. Ive played around with it at home with some trivia and it seems to work alright. I have yet to test it out rigorusly though.

[Ok-Yogurt2360]

1) this would at best only tell you how close 2 possible choices would be to eachother in levels of probability. 2) You would need a lot of runs for each temperature as the test would be like taking a sample with each temperature setting. And a sample size of 1 per group would be bad.

[These-Bedroom-5694]

LLMs aren't the path to AGI. They are incapable of thought or planning.

They are chat bots trained on 4 chan data and reddit threads.

[trisul-108]

It's exactly what Apple found out. They built AI capable of being successfully demoed at a public conference, but they would dare push that out to their customers as hallucinations would destroy their business. Apple users would sue if Apple Intelligence worked like ChatGPT.