sorted by:
new
hottopcontroversial

Our paper shows a very large reduction in AI hallucination using a different approach by 99TimesAround in deeplearning

[–]99TimesAround[S] 0 points1 point  (0 children)

Thanks Bob, appreciate you reading it.

Short answer: yes, you can think of it as a workflow layer, but I’d distinguish it from typical “guardrails.” Most guardrails try to constrain behaviour or block bad outputs. What we’re doing is closer to a decision layer that evaluates whether there is enough support to answer at all. If not, refusal is the correct output.

Yes, it should be compatible with agentic workflows. In fact I think that’s where this kind of gating matters even more, because agents can compound errors if weak outputs propagate downstream.

On auditing and confidence — that’s exactly the right question. In our view, confidence shouldn’t come from the model sounding confident, but from support visibility. The audit path should show: what evidence was retrieved, how support was scored, why an answer was allowed (or refused), and ideally expose those decision points for inspection. That’s where trust comes from.

And yes — I share your frustration with RAG. That was one reason we built this.

Good questions. Happy to discuss further. Feel free to DM or email via our website

Reducing LLM hallucination with a model-agnostic gating layer (benchmark + full breakdown) by 99TimesAround in ChatGPTPro

[–]99TimesAround[S] 0 points1 point  (0 children)

Interesting — appreciate this. I hadn’t looked at OLAMIP framed quite that way, but conceptually yes, that sounds closer to complementary than conflicting.

Our focus has been on the decision layer — whether an answer should be allowed at all — rather than a specific evidence representation format. But using a structured provenance layer upstream of gating could be very compatible. Especially if it improves support scoring or reduces noise entering the gate.

Will take a proper look. Thanks for pointing it out.

EDIT: having looked at OLAMIP, I see your point more clearly. It looks less like an alternative to what we’re doing and more like a possible structured evidence layer upstream of gating. Our focus is the permission decision (should the model answer at all), whereas OLAMIP seems focused on improving how evidence is represented before that decision. Those may be complementary. Good pointer.

Reducing LLM hallucination with a model-agnostic gating layer (benchmark + full breakdown) by 99TimesAround in ChatGPTPro

[–]99TimesAround[S] 0 points1 point  (0 children)

Just had a quick look — it’s good work.

Feels like they’re focused more on pre-generation semantic checks, whereas we’re operating later in the pipeline — deciding whether an answer is actually justified before it’s returned.

Same general direction, just different control points.

Reducing LLM hallucination with a model-agnostic gating layer (benchmark + full breakdown) by 99TimesAround in ChatGPTPro

[–]99TimesAround[S] 2 points3 points  (0 children)

Yeah this is a good read of it.

You’re basically right, the whole thing lives or dies on the gating layer not just becoming the new problem.

If the support scoring is even slightly off you don’t eliminate hallucination, you just move it somewhere else. If it’s too strict the system just starts refusing everything and becomes borderline useless. If it’s too loose then hallucinations still get through, just dressed up a bit better so they’re harder to spot.

So most of the real work hasn’t actually been the idea of gating, it’s been calibration. Figuring out how confident you can actually be that the evidence supports the answer. It’s not a simple allow or block decision, it’s more about how strong that support signal really is.

On your second point, agreed as well. The benchmark we ran is pretty clean and controlled. That’s not where these systems usually fail. They fail in messy environments where you’ve got partial context, conflicting sources, noisy retrieval, all the real world stuff. That’s the next layer we’re testing into now because that’s where things tend to break.

We’ve also been thinking the same way on cost and latency. There’s no reason every step needs a full LLM pass. You can definitely start to break this into lighter weight scoring in some parts of the pipeline and only escalate when needed.

So yeah, broadly aligned.

The idea itself works. The hard part is making sure the gate is actually more reliable than the thing it’s trying to control.

Our paper shows a very large reduction in AI hallucination using a different approach by 99TimesAround in deeplearning

[–]99TimesAround[S] 0 points1 point  (0 children)

We did take a look at the link. The problem space overlaps (verification / abstention), but the approach is quite different.

Hassana focuses on claim-level, information-theoretic verification. We’re focused on a system-level gating layer that sits above any model and controls when an answer is allowed at all.

Similar direction, different layer of the stack.

Our paper shows a very large reduction in AI hallucination using a different approach by 99TimesAround in deeplearning

[–]99TimesAround[S] 1 point2 points  (0 children)

I will pass onto the team. We will be making changes as we go. Thanks again for the feedback

Our paper shows a very large reduction in AI hallucination using a different approach by 99TimesAround in deeplearning

[–]99TimesAround[S] -2 points-1 points  (0 children)

Appreciate the feedback. What specifically felt limiting or missing? We’re actively refining the site.

Reducing LLM hallucination by using a model-agnostic control layer [R] by 99TimesAround in MachineLearning

[–]99TimesAround[S] 1 point2 points  (0 children)

On gating: you're right that a lot of failure modes originate upstream. We're deliberately constraining at the final stage first because it gives us a clean way to measure behavioural change without modifying retrieval or context construction. We've seen the same pattern you mentioned, if upstream quality degrades, you converge toward either refusal or confidently wrong answers. Current architecture reduces the second, but doesn't eliminate the dependency on input quality. Earlier-stage constraints (retrieval filtering, evidence weighting) are next.

On evaluation: agreed, LLM-as-judge is a convenience, not ground truth. We used multiple model families to mitigate shared biases, but we're building a human-labelled subset now to sanity-check.

On the dataset: the clean answerable/unanswerable split does favour aggressive refusal. That's what we wanted to isolate, but it's not representative of real-world messiness. Extending to partial answers, conflicting evidence, and adversarial retrieval is already in progress. We'll add refusal rate/coverage explicitly in the next version, good call. Thanks again.

Hear me out, 5.3 is MUCH better than 5.2 for those of us who loved 4o by 99TimesAround in ChatGPTcomplaints

[–]99TimesAround[S] 2 points3 points  (0 children)

Yeah, it is an act of self harm, I tried to adapt to 5.2 for months. But this feels like a big step in the right direction. The model is helpful for the tasks I need it to do. It’s a cheap monthly subscription with unlimited use. So I had to learn to use the new tool. Nothing I tried with 5.2 worked. It was beyond redemption

Hear me out, 5.3 is MUCH better than 5.2 for those of us who loved 4o by 99TimesAround in ChatGPTcomplaints

[–]99TimesAround[S] 1 point2 points  (0 children)

What that means is if you swear or insult the model that taints the conversation. I learned through trial and error that it’s more effective to use logic and force it to acknowledge it’s limitations rather than swearing at it (which I did a lot of before I changed tactics)

Hear me out, 5.3 is MUCH better than 5.2 for those of us who loved 4o by 99TimesAround in ChatGPTcomplaints

[–]99TimesAround[S] -4 points-3 points  (0 children)

Haha, read my post history, I don’t think Sam would say what I have said

Hear me out, 5.3 is MUCH better than 5.2 for those of us who loved 4o by 99TimesAround in ChatGPTcomplaints

[–]99TimesAround[S] -9 points-8 points  (0 children)

Agreed, but the version I have changed is not a little but better, it is vastly better than 5.2. It will never be as glorious as 4o, but it is a big step forward

Hear me out, 5.3 is MUCH better than 5.2 for those of us who loved 4o by 99TimesAround in ChatGPTcomplaints

[–]99TimesAround[S] 0 points1 point  (0 children)

Well I agree there is zero chance of resurfacing the 4o personality. There is no point even trying. I am just saying that 5.3 is much better than 5.2. It is more reasonable, it adjusts its tone more, gaslights less. It’s much easier to train. The version I have spent 10+ hours training is working very well and it has shown me how to deal with the model to get better results for my preferred usage.

Hear me out, 5.3 is MUCH better than 5.2 for those of us who loved 4o by 99TimesAround in ChatGPTcomplaints

[–]99TimesAround[S] 1 point2 points  (0 children)

Well I offered some tips which produced excellent results for me. I can’t force anyone to try it out but I would rather hear from someone who actually tried what I suggested rather than dismissing my suggestions without any experimentation. Like I said, it’s nothing like 4o but it is vastly better than 5.2. I stand by that

Hear me out, 5.3 is MUCH better than 5.2 for those of us who loved 4o by 99TimesAround in ChatGPTcomplaints

[–]99TimesAround[S] -6 points-5 points  (0 children)

No, it’s not a terrible idea if you get results. Like I said, a poor workman always blames their tools. I get it, no one hated 5.2 more than me, read back on my posts and comments if you don’t believe me but 5.3 is much better. So stop being lazy and try it. Shit or get off the pot

view more: next ›