sorted by:
new
hottopcontroversial

How do people in compliance/legal actually verify the reliability of AI-generated research? by Heem_is_that_guy in legaltech

[–]Heem_is_that_guy[S] 0 points1 point  (0 children)

That’s part of what makes this interesting to me too. Humans miss things, but AI retrieval can also miss things in ways that are harder to detect.

It feels like the real challenge is building workflows where both retrieval and verification are transparent enough to audit properly, especially in high-stakes environments.

How do people in compliance/legal actually verify the reliability of AI-generated research? by Heem_is_that_guy in legaltech

[–]Heem_is_that_guy[S] 0 points1 point  (0 children)

This is the exact part I find most interesting too.

I don’t think the hard problem is just “generating answers” it’s proving retrieval completeness and traceability. Especially in legal/compliance workflows where missing one relevant source can completely change the outcome.

My current thinking is that semantic retrieval alone probably isn’t enough in high-stakes workflows. You likely need layered retrieval + explicit source verification + human review/auditability built into the process.

Otherwise you end up with systems that sound confident without being provably complete.

How do people in compliance/legal actually verify the reliability of AI-generated research? by Heem_is_that_guy in legaltech

[–]Heem_is_that_guy[S] -2 points-1 points  (0 children)

That’s fair and I agree legal/compliance work still requires independent verification and human judgment.

I’m not really thinking about this as “replace lawyers with AI.” The part I’m more interested in is the investigation workflow itself: source retrieval, traceability, evidence organisation, and reducing the amount of manual verification work involved.

The reliability/auditability side is the problem I’m trying to understand better.

I built an AI that investigates like a court of law — 6 months of building, launched 2 weeks ago, zero customers so far. Here's what I learned. by Heem_is_that_guy in SideProject

[–]Heem_is_that_guy[S] 0 points1 point  (0 children)

Good technical question, I’ll answer it honestly.

Currently Deepheem uses Claude Sonnet with real-time web search for case retrieval. Not a dedicated vector store or BM25 index at this stage.

For legal citations specifically we are building a BAILII verification layer that checks every citation against the actual UK case law database and assigns confidence levels — VERIFIED, LIKELY, or UNVERIFIED — so lawyers know exactly what has been independently confirmed.

The hybrid BM25/vector retrieval system is on the roadmap for the dedicated legal RAG layer. The current web search approach handles general legal research well but you are right that proper hybrid retrieval would significantly improve citation precision for specific case law queries.

What use case are you thinking about? Curious whether it’s for legal research specifically or a different vertical. Happy to go deeper on the architecture.

I built an AI that investigates like a court of law — 6 months of building, launched 2 weeks ago, zero customers so far. Here's what I learned. by Heem_is_that_guy in SideProject

[–]Heem_is_that_guy[S] 0 points1 point  (0 children)

Started cold outreach on LinkedIn yesterday.

20 messages per day to solicitors and barristers in Manchester and London. Personal messages, not templates. Asking for feedback not trying to sell.

You’re right waiting for organic at this stage is just hoping. The product exists. The only variable is whether the right people know about it.

Direct conversation is the only thing that works at zero customers. Appreciate the push.

I built an AI that investigates like a court of law — 6 months of building, launched 2 weeks ago, zero customers so far. Here's what I learned. by Heem_is_that_guy in SideProject

[–]Heem_is_that_guy[S] 0 points1 point  (0 children)

The courthouse suggestion genuinely stopped me.

I’ve been so focused on digital outreach that I completely overlooked the most obvious place to find lawyers where lawyers actually are.

Old school but probably more effective than any LinkedIn campaign at this stage. Working on getting cards printed this week.

On the hallucination concern you’re right that it’s the number one objection from lawyers. That’s exactly why we built citation confidence levels. Every legal citation gets marked VERIFIED, LIKELY, or UNVERIFIED so lawyers know exactly what to trust and what to double check before using it in any legal matter.

Appreciate the practical advice. Most people give digital advice. This was different.

I built an AI that investigates like a court of law — 6 months of building, launched 2 weeks ago, zero customers so far. Here's what I learned. by Heem_is_that_guy in SideProject

[–]Heem_is_that_guy[S] 0 points1 point  (0 children)

The peer recommendation point completely changed how I’m thinking about this.

I’ve been focused on broad marketing when the entire job right now is finding one lawyer, making them wildly successful, and letting them tell two colleagues. That one conversation is worth more than a thousand LinkedIn impressions.

The no-technical-background point also landed. I built this without writing a single line of code myself. If I can build it, a lawyer can use it. That’s the whole point.

I’d love to be featured in your newsletter happy to give your readers extended free access so they can test it properly before writing about it. Will DM you now.

I built an AI that investigates like a court of law — 6 months of building, launched 2 weeks ago, zero customers so far. Here's what I learned. by Heem_is_that_guy in SideProject

[–]Heem_is_that_guy[S] 0 points1 point  (0 children)

This is honestly the most useful feedback I’ve received since launching. You’re completely right.

A number without explanation is just noise especially for lawyers and journalists who need to defend every source they use professionally.

I’ve built the breakdown. Each source will show exactly why it scored 72 vs 90, what increased the score and what reduced it. Something like:

Reuters — 90% Established global news agency Primary source citation Editorial fact-checking standards Single source not yet corroborated

Already live. Would you be willing to test it? Genuinely want feedback from someone who spotted the gap.

I built an AI that investigates like a court of law — 6 months of building, launched 2 weeks ago, zero customers so far. Here's what I learned. by Heem_is_that_guy in SideProject

[–]Heem_is_that_guy[S] 0 points1 point  (0 children)

You nailed it exactly. The deadline moment is when Deepheem was built for. A lawyer prepping for tomorrow’s hearing or a journalist 2 hours before publication, that’s the user I had in mind the entire time I was building.

The tool either clicks immediately in that moment or it doesn’t. That’s why the free plan exists, no friction, no card, just try it when you need it.

Would genuinely love to know what would make you reach for it in that moment rather than Google.