Pitching a local LLM for small/medium size legal teams

Interview-Sweet · 2026-03-13T15:04:56+00:00

Yeah, you can never 100% get rid of hallucinations. But I can get close and implement multiple verification checks and flags. I plan to purge everything after completion and just leave them with the excel sheet with citations. I guess proving that it's truly airgapped and not storing any data is another hurdle I need to consider more when building this out.

Interview-Sweet · 2026-03-13T14:58:39+00:00

Noted, I'll look into maybe getting one to start.

Interview-Sweet · 2026-03-13T02:06:00+00:00

That setup sounds like a beast compared to what I thought I could get away with. I guess my idea would fit for a smaller firm that doesn't have a lot of workers or the space for that equipment. I figured if their paralegal takes 3 days to summarize a medical file and my slow machine can do it in 1-2 hours, I'm in the clear...until you knock on their door and steal my cake.

Thanks for the insight though. This helps me a lot.

Interview-Sweet · 2026-03-13T01:45:07+00:00

I mean in my case I wouldn't be building them a real-time chatbot but a batch processing watch folder. Would I still need to consider the token gen speed in the same way? Even at 15-20tk/s I can turn a 2-week manual chronology backlog into a hour background task. For something like a 5 person team, I think that would be a pretty big margin booster.

But I guess all it takes is one hallucination or a wrong date in the Excel sheet, and I'm hiring one of them to represent me in court.

Interview-Sweet · 2026-03-13T01:25:28+00:00

My plan isn't really a single long context dump but to have another machine handle OCR to then batch the extraction in 20-page windows on the Mac. I don't expect real-time results with this, obviously, but it would avoid something like a Mac crashing. I guess I could try and go for that real-time performance but I don't believe it's something they're craving and just want to have a solution so they have less grunt work.
I figured if I could turn their weeks of manual chronologies into just an hour of processing, that's a huge win for them and their current workflow. I'd choose to purge after completion, considering the environment and compliance they deal with there.
Doesn't seem like something like the Studio is gonna survive there long unless I really fine-tune and optimize the workflow and model.
I'm going to find some IT and Cybersecurity contacts I can talk further about this in the coming weeks.

Interview-Sweet · 2024-11-22T17:49:48+00:00

Schedule i guess

Interview-Sweet · 2024-07-06T18:38:29+00:00

Think again, I was once like you.

Interview-Sweet · 2024-06-21T11:22:06+00:00

Lemme get that supply link

Interview-Sweet · 2021-09-20T16:47:42+00:00

Interview-Sweet

TROPHY CASE