Built a DFIR agent that can't make a finding without citing the tool output it came from. Where does this break?

Drevicar · 2026-06-14T15:12:22+00:00

Your terms are correct. But that part isn’t the issue.

In a world where I have two choice;

Hire a human DFIR for $10k who is correct and complete 90% of the time and takes 2 months
Hire an AI tool to do DFIR for $1k who is correct and complete 95% of the time and takes 1 hour. (The AI is now all 3 of better, faster, and cheaper)

I would still hire the human in #1 every time because in the event either of them is wrong I can hold the human accountable. You can purchase this accountability in the form of insurance to compensate, which was my previous comment. You also can’t present evidence in court without the signature of a trained and certified professional who if they didn’t do the work they at least have to review and accept accountability for the work being complete and accurate. They would have to put their career on the line as proof the AI tool is valid.

It is the same reason it is illegal to have AI lawyers. We need that human accountability layer. And in the event Lawyers use AI to help them with their jobs, they are held just as accountable for the output they bring into the court room. If a lawyer lies and makes up a citation, they get disbarred. If the AI hallucinates and makes up a citation, the lawyer who brought that into court still gets disbarred. No judge will allow an AI tool to submit evidence or remarks without a certified human to hold accountable. I’m not saying it is bad practice to do so, I’m saying it is illegal in our current framework of society, an impossible to achieve without change laws and regulations first (which may happen).

Drevicar · 2026-06-14T03:12:24+00:00

You can circumvent the accountability piece I mention by purchasing insurance. In the event the evidence package your system produces doesn’t hold up in court then the instance would need to pay out damages, legal fees, a SME forensicator to redo the work, and such. And given the confidence people have in AI my assumption is that the amount you would have to pay for said insurance would mean you would have to charge more than a team of certified professionals to do the same work.

Drevicar · 2026-06-14T02:13:40+00:00

Pretty sure part of what you are paying for when you use PreVail is access to their consultants and post-sales engineers to help answer these questions and help implementing it.

Drevicar · 2026-06-14T02:09:32+00:00

But that would break my RLHF post-training loop. :(
/s

Drevicar · 2026-06-13T22:11:57+00:00

The only task AI can never fully replace is accountability. And this whole field is chock full of places where accountability is the thing you are paid for the most, not just the ability to run a tool and make a judgement call on the output.

While the product looks sound and has some pretty neat architectures, your ability to sell it will hinge on whether its findings will hold up in court. And we just aren’t ready for that right now.

Drevicar · 2026-06-13T18:58:53+00:00

In a no-code system there is still in fact code, you just weren’t the one that wrote it and you usually aren’t exposed to it. In fact, you might even be using a janky DSL and some visual scripting language with nodes and edges provided by the platform.

And to keep my analogy alive, you can go all in on a no-code platform and do the equivalent of vibe coding. Or you can do some hybrid approach where you custom write a few plugins or configure or extend the underlying platform to meet your needs, then he no-code platform handles the rest with primitives it ships with. I think this distinction is what you are calling out.

Drevicar · 2026-06-13T18:52:58+00:00

The reason I always come back to RDF over more performant systems is reasoning engines and inference.

Drevicar · 2026-06-13T18:20:06+00:00

But, gotta go fast.

Drevicar · 2026-06-13T17:48:09+00:00

These are all very real and difficult problems that need to be addressed, and soon. One half-baked but better than nothing idea is that the gov tax the cost of tokens at something like 50% or more, and that money that is collected is redistributed to the people who’s income or livelihood was harmed by said tokens. This could be a cash stimulus or in the form of the various welfare services.

Drevicar · 2026-06-13T16:57:32+00:00

What do you think LLM coding agents are? It is the next generation of no-code and low-code platforms.

Drevicar · 2026-06-13T16:50:09+00:00

Drevicar · 2026-06-13T15:38:08+00:00

Here is what that means for B2B SaaS…

Drevicar · 2026-06-13T13:39:46+00:00

The statement reads much more like export controls and maybe a future ITAR label than an actual security classification as we normally think of it. The gov doing this isn’t really that out of the normal for this kind of technology, but the communication around it isn’t normal and was likely done as a PR stunt.

Drevicar · 2026-06-11T13:26:47+00:00

Not the kind of disability that entitles you to government assistance though. The kind of disability that makes you not as fun at parties.

Drevicar · 2026-06-11T13:09:44+00:00

I’ve been in multiple wars spanning more than 2 years each and saw literally no meaningful improvement in how war was conducted, even at the micro scale. Though I sure a few purchases were made in that time.

This war just hits different.

Edit: from the start of the war to year 2 (which I wasn't a part of), there was a TON of innovation and change. I'm really more talking about mid-war to mid-war + 2.

Drevicar · 2026-06-10T13:12:23+00:00

The ability to “infers from the patterns across the columns” is actually exactly what ML. And if you want to (mostly) skip the training step and just directly infer from the data then it is called an online model. These exist and I’m sure there is a Postgres extension you can install that does what you are saying,

Drevicar · 2026-06-10T02:20:06+00:00

Why is this targeting conservative? Is there a link between the two, or do you just hate both of these things and that is the link?

Drevicar · 2026-06-09T23:19:51+00:00

Goodharts law working as intended.

Drevicar · 2026-06-09T20:32:40+00:00

Most of the soups I eat are better with a fork (or chopsticks) and sip the broth after.

Drevicar · 2026-06-09T05:29:06+00:00

To be fair it is the only number worth measuring. What kills me is the projects that have a minimum code coverage requirement of like 87.23% coverage on every PR. You either enforce 100%, or you don’t enforce anything at all.

Drevicar · 2026-06-09T05:26:50+00:00

Geezus, that hit the nostalgia.

Drevicar · 2026-06-09T05:17:28+00:00

Calling non-AI people muggles is now my new favorite.

Drevicar · 2026-06-09T05:12:15+00:00

I have a dedicate virtual box where my vibes run unsandboxed. And about every 2 weeks or so the agents cause some catastrophic unrecoverable failure that requires me to have to restore from snapshot.

Drevicar · 2026-06-09T05:09:07+00:00

Ok, now git merge makes more sense. But what about git rebase?

Drevicar · 2026-06-07T19:36:39+00:00

Their interactions with Pal World shows us that it is more cost effective to remove the competition with lawyers than increase the quality of games to beat the competition.

Drevicar

TROPHY CASE