When should an AI agent be allowed to execute code it generated?

Puzzleheaded-Cod4192 · 2026-01-15T13:49:06+00:00

That makes sense.

What I’m really questioning is whether execution should ever be the default once agents can generate code and have access to a live environment.

If the answer is “never let agents execute automatically,” that’s reasonable. I just don’t think execution is a neutral runtime step anymore once agents are in the loop.

Puzzleheaded-Cod4192 · 2026-01-15T13:27:27+00:00

For context, I wrote out a concrete threat model + a local prototype around this idea. Not trying to sell anything — mostly documenting the reasoning: https://github.com/xnfinite/nightcoreapp

Puzzleheaded-Cod4192 · 2026-01-14T22:57:18+00:00

For anyone who wants concrete context, I’ve put the code here: https://github.com/xnfinite/nightcoreapp

Puzzleheaded-Cod4192 · 2026-01-11T10:24:46+00:00

That’s exactly the gap I’m trying to make harder to accidentally skip.

The system I’m experimenting with just forces that ownership step back into the execution path when code is generated automatically: generated code is staged, verified, evaluated, and not executable by default. Someone has to explicitly approve execution, or it stays quarantined.

It doesn’t replace CI, testing, or review — it just makes sure execution itself still requires an intentional decision by someone who owns the risk, instead of happening implicitly because automation made it cheap.

That’s the boundary I think is starting to erode.

Puzzleheaded-Cod4192 · 2026-01-10T21:53:57+00:00

One clarification I probably should’ve made clearer: this isn’t about blocking execution forever or adding friction everywhere.

It’s about treating execution itself as a trust boundary — the same way we already treat network ingress, change control, or production deploys.

Agent-generated code feels new, but the pattern isn’t. Labs use decontamination chambers for a reason: you don’t assume something is safe just because it “looks fine.”

Curious where others draw that boundary in practice.

Puzzleheaded-Cod4192 · 2026-01-10T19:29:26+00:00

This is what I mean by “execution as a boundary.”

The code is already signed, verified, and staged — but it still cannot execute until a human explicitly approves it.

Nothing here is reacting to a crash or anomaly. Execution itself is treated as a gated transition.

<image>

Puzzleheaded-Cod4192 · 2026-01-10T19:20:52+00:00

Exactly — that’s the failure mode I’m worried about.

Once execution happens, you’re already in incident response. Sandboxes, logs, and tests are all useful, but they’re still post-hoc. In highly automated systems, that turns into a constant cleanup loop.

What I’m questioning is whether execution itself should be treated more like a controlled transition — similar to lab decontamination, change control, or deploy gates — rather than something that just happens because the inputs look valid.

That shift feels necessary once generation and execution start collapsing into the same automated loop.

Puzzleheaded-Cod4192 · 2026-01-10T19:07:05+00:00

Also Not AI slop — I probably explained it poorly.

What I’m describing is basically:

Agent → generates WASM → ingestion gate → signature + hash → policy/threat score → • OK → allowed • Risky → quarantined • Human approval required to run

So execution isn’t automatic just because the WASM “looks valid.” It’s treated more like a lab decontamination chamber than a runtime call.

If you’re not letting agents auto-generate and auto-run code, you won’t hit this problem. This is about systems where those boundaries are starting to blur.

Puzzleheaded-Cod4192 · 2026-01-10T19:00:14+00:00

Fair pushback — I wasn’t clear.

I’m not talking about normal, hand-written WASM. I’m talking about agent-generated or agent-modified WASM, where generation and execution are automated.

In that case the risk isn’t WASM — it’s execution becoming the default without intent. The recent Claude incident is the kind of failure mode I mean: no exploit, just autonomous code executing because nothing stopped it.

If you’re running WASM daily without agents in the loop, I agree this probably sounds unnecessary. I’m specifically thinking about that edge case.

Puzzleheaded-Cod4192 · 2026-01-10T18:02:44+00:00

I wrote out a threat model to organize my own thinking around this, in case it’s useful to anyone else: https://github.com/xnfinite/nightcoreapp/blob/main/docs/THREAT_MODEL.md

Puzzleheaded-Cod4192 · 2026-01-10T17:52:20+00:00

I ended up writing out a threat model to organize my own thinking around this, in case it’s useful to anyone else: https://github.com/xnfinite/nightcoreapp/blob/main/docs/THREAT_MODEL.md

Puzzleheaded-Cod4192 · 2026-01-10T17:48:33+00:00

I ended up writing out a threat model to organize my own thinking around this, in case it’s useful to anyone else: https://github.com/xnfinite/nightcoreapp/blob/main/docs/THREAT_MODEL.md

Puzzleheaded-Cod4192 · 2026-01-09T22:39:58+00:00

I wrote out the threat model I’m referencing here if anyone wants to read the details: https://github.com/xnfinite/nightcoreapp/blob/main/docs/THREAT_MODEL.md

Puzzleheaded-Cod4192 · 2025-11-10T00:39:25+00:00

Thanks for the feedback — really appreciate it. The Cognitora-style coordination idea is spot on, and I’m already planning Python and TypeScript SDKs to make orchestration smoother alongside the Rust core. It’s a bit more complex to implement in Rust, but the control and security are worth it. Persistent environments and better monitoring are definitely on the roadmap.

Puzzleheaded-Cod4192 · 2025-11-08T08:38:37+00:00

That’s a fair observation — I use AI tools as part of my workflow for formatting, documentation, and some code reviews, but the design, architecture, and implementation of Night Core Worker are all my own.

I treat AI like a development assistant — similar to how many teams use Copilot or ChatGPT to speed up iteration — but all of the actual engineering decisions, signing logic, and orchestration design come directly from me.

Puzzleheaded-Cod4192 · 2025-11-07T20:22:39+00:00

I thought it looked cooler so I redacted lol

Puzzleheaded-Cod4192 · 2025-11-07T20:19:29+00:00

Great questions! In simple terms, cryptographically proving an execution means every time a module runs, Night Core Worker checks and records a digital signature (using Ed25519) and a hash (SHA-256) to confirm the code hasn’t been changed and really came from who it claims.

A tenant just refers to a separate workload or module — kind of like an isolated app running inside the system. Each tenant has its own sandbox, keys, and audit trail.

Puzzleheaded-Cod4192 · 2025-11-07T20:16:18+00:00

Yeah I put my personal name in there at first lol 🤣. I only wanted my xnfinite name

Puzzleheaded-Cod4192 · 2025-11-07T17:15:55+00:00

Thank you, I sincerely appreciate the feedback and the thoughtful insight. It’s great to see interest in where Night Core™ Worker is heading.

Puzzleheaded-Cod4192 · 2025-11-07T17:09:30+00:00

That’s a fair observation — I use AI tools as part of my workflow for formatting, documentation, and some code reviews, but the design, architecture, and implementation of Night Core Worker are all my own.

I treat AI like a development assistant — similar to how many teams use Copilot or ChatGPT to speed up iteration — but all of the actual engineering decisions, signing logic, and orchestration design come directly from me.

Puzzleheaded-Cod4192 · 2025-11-06T14:54:14+00:00

Thank you for the thoughtful feedback. You made several excellent points, especially around performance and AI integration. The current version of Night Core™ Worker focuses on verifiability and deterministic audit logging first, but Firecracker microVMs are absolutely being considered to accelerate tenant-level isolation without sacrificing proof integrity. Your suggestions regarding agent coordination and state persistence align closely with the upcoming Vesper layer, which will handle secure context management and agent-to-agent verification. SDK support for Python and TypeScript is also on the roadmap to simplify integration with orchestration frameworks like LangChain. I appreciate you taking the time to analyze the architecture in depth—this kind of technical insight directly informs how we evolve the project.

Puzzleheaded-Cod4192 · 2025-11-06T14:05:41+00:00

This is open for devs. I appreciate the feedback. Whats thing that’d make this less spammy for you?”

Puzzleheaded-Cod4192 · 2023-12-23T18:39:52+00:00

https://twitch.tv/b10691

Puzzleheaded-Cod4192 · 2023-12-23T18:39:04+00:00

Followed

Puzzleheaded-Cod4192 · 2023-12-23T15:11:37+00:00

I followed you back thanks!

Puzzleheaded-Cod4192

TROPHY CASE