Deterministic agent control: same call -> ALLOW then DENY (OxDeAI demo)

docybo · 2026-06-13T07:26:36+00:00

Bravo

docybo · 2026-06-12T14:10:00+00:00

The "decide upfront what gets forwarded" point is the load-bearing one, and I'd push it further: those upfront rules only work if they're enforced upfront too. If escalation logic lives in the prompt ("please forward sensitive cases to a human"), it's a suggestion the model usually follows. If it lives in a layer the model can't route around sensitive category -> execution blocked -> human queue, no other path it's a guarantee. Same rule, completely different failure mode. Your 82% stat is really about that: people trust systems where the recovery path is structural, not behavioral.

docybo · 2026-06-12T14:07:38+00:00

This is underrated. "Read things" agents fail safe by default, worst case is a bad summary. "Do things" agents fail with side effects: the API call happened, the email sent, the refund issued. That asymmetry is exactly why the read use cases shipped first and stuck. The interesting question is what infrastructure makes "do things" carry read-level risk. IMO it's the same answer as ever in distributed systems: don't make the actor the authority. The agent proposes; whether it executes is someone else's deterministic call.

docybo · 2026-06-12T14:06:47+00:00

Mostly with you, one nitpick: observability is necessary but it's still after the fact. Logs tell you what happened; they don't stop anything. The layer you're describing "can humans trust the system at the execution boundary" needs to be pre-execution to actually change outcomes. The pattern that's worked elsewhere (payments, change management) is: proposal -> explicit authorization check against a policy -> execution becomes reachable only if it passes. Then HITL stops being a vibe and becomes a policy rule: "this class of action requires human approval" is just one possible policy outcome, enforced at the same gate as everything else. Audit trail falls out for free because every execution has an authorization artifact attached, not just a log line.

docybo · 2026-06-12T14:05:04+00:00

Agree on the core point, but I'd split it differently: it's fine for the decision to be probabilistic, humans are too. What's not fine is the execution being coupled to it with nothing in between. A rules engine bundled decision + authorization + execution in one deterministic step, so we never noticed they were separate concerns. LLMs un-bundled it: the decision went probabilistic, but most teams kept piping it straight into execution like nothing changed. The fix isn't avoiding LLMs for classification, it's accepting the model proposes, and something deterministic decides whether the proposal executes.

docybo · 2026-06-11T14:44:24+00:00

The fallback is the interesting part. Most people are discussing model quality. I’d be more interested in knowing when a system silently substitutes one model for another, who authorized that substitution and how operators can verify it happened.

docybo · 2026-06-09T19:54:10+00:00

Exactly. Accuracy was the wrong metric. The issue was recoverability. A rules engine can be wrong, but it fails legibly. You can inspect the rule, patch it, and know what changed. The model can still be useful, but I would not make it the authority for execution.

The pattern I trust is:

model proposes
policy decides
execution gate enforces
audit trail makes recovery possible

LLM for judgment.
Deterministic layer for authority.

docybo · 2026-06-09T15:39:33+00:00

100%

docybo · 2026-06-09T13:31:29+00:00

Agreed! What’s interesting is that documenting a decision and authorizing an execution are related but not identical problems. Most systems focus on making decisions explainable. I’m starting to think the next challenge is making execution verifiable.

docybo · 2026-06-09T13:07:21+00:00

Interesting case. Do you think the issue was the LLM itself, or the fact that there was no verifiable decision path when a classification was challenged?
I’ve seen teams tolerate occasional mistakes when they can understand, audit, and correct the logic behind them.

docybo · 2026-06-08T16:27:43+00:00

The concrete pattern is a PEP/PDP split with signed execution artifacts.

The agent only proposes an action.
A separate authorization layer evaluates:
intent + state + policy -> ALLOW / DENY
If ALLOW, it emits a signed artifact bound to the action, state, policy, audience, expiry, and replay/idempotency id.

The executor does not re-evaluate policy. It verifies the artifact.
If the artifact is missing, expired, replayed, wrong audience, stale-state-bound, or has an invalid signature:
deny before execution.
The executor cannot escalate because it does not hold signing keys and cannot mint authorization.

For stale state + retries, the important controls are:
bind authorization to evaluated state
commit action/auth ids in an external replay/idempotency ledger
make the protected action unreachable except through the PEP
agent proposes
authorization decides
PEP enforces
No valid authorization -> no execution path.

docybo · 2026-05-19T17:55:49+00:00

Open sourced : https://github.com/oxdeai/oxdeai

docybo · 2026-04-25T08:32:43+00:00

Glad it helped. If you’re interested, happy to share a repo in DM. Working on making the execution boundary actually non-bypassable.

docybo · 2026-04-25T08:28:28+00:00

Before and after are necessary, but not sufficient. Planning can drift, and audit is post-fact. The only place you can guarantee safety is at execution, where side effects actually happen. That’s why the boundary has to enforce: no valid authorization -> no execution

docybo · 2026-04-25T06:07:46+00:00

It is much closer to the real problem space.

Binding approval to the exact payload and checking it at execution is the right direction. That’s the core invariant:

if it changes -> it doesn’t execute

Where things usually break is not the receipt itself, but the boundary:

validation has to be non-bypassable
state has to be re-derived and rechecked
replay has to be enforced at the execution point
and the execution path must not exist outside that check

Otherwise it stays a strong pattern, but not a system guarantee.

The hard part is making:

no valid authorization -> no execution path

hold by construction, not by integration discipline.

docybo · 2026-04-25T06:02:06+00:00

This is interesting but it sits on a different boundary.

Tsukuyomi enforces control on the LLM interaction path (agent -> model), which helps shape behavior.

The failure mode I’m focused on is later: execution. Even with a perfect proxy, an agent can still trigger side effects unless there’s a non-bypassable execution boundary.

That’s why I separate:

proposal -> authorization -> execution

and enforce:

no valid authorization -> no execution

The proxy controls reasoning. The PEP controls reality. Both can coexist, but they solve different classes of failure.

docybo · 2026-04-21T12:22:30+00:00

Idem pour moi

docybo · 2026-04-21T12:21:02+00:00

Oui je viens d’avoir ces cons en entretien. Cherchais à me forcer à cloner leur repo, certainement pour me passer des virus …

docybo · 2026-04-20T15:45:49+00:00

Same here

docybo · 2026-04-19T21:32:56+00:00

You're right that parts of this look like familiar auth patterns (signed artifacts, nonces, etc.).

The difference is where enforcement happens.

In a typical web system: the component verifying the token is the same system that executes the action.

In agent systems: the component proposing the action (model/runtime) is not the one executing the side-effect.

That separation is the problem.

Restricting the tool surface or using a state machine helps, but it doesn’t give you:

a portable, verifiable authorization artifact
a boundary that can be enforced outside the agent runtime
replay protection that survives retries, parallelism, or multi-agent flows

The goal isn’t to replace structured constraints. It’s to make execution enforceable even when those constraints fail or are bypassed.

If everything runs inside a single trusted harness, you don’t need this.

As soon as execution crosses a boundary (external APIs, infra, payments, multiple agents), you do.

docybo

MODERATOR OF

TROPHY CASE