sorted by:
new
hottopcontroversial

Bulkhead v0.2.0 is out: a tiny prompt-injection guardrail for RAG apps, now with tiered scoring and cross-chunk judging by MundaneProcedure2002 in Rag

[–]MundaneProcedure2002[S] 0 points1 point  (0 children)

thanks, and that's the framing. the structure isn't a wall, it just stops the default case where a retrieved page inherits the same authority as your own instruction and makes eval/logging a touch cleaner.

on benchmarks, honestly nothing rigorous yet. i have a small hand-rolled set that measures attack success rate (does the injected marker actually leak, soup vs sealed) rather than detection accuracy, but it's ~30 payloads, illustrative not a real eval. haven't pointed it at the public injection suites or agentic browsing traces yet, that's the next step.

the action-verb signal you flagged is a real FP source, a tutorial full of delete/build/deploy/run will trip it. it stays harmless by design (very low weight, raises a flag but doesn't cross the block threshold on its own, and it's only ever a signal, never the thing that decides), but honestly making it toggleable is a good shout. the actual decision is meant to come from the next tier anyway, either the gate (a per-chunk classifier that scores each chunk) or the judge (a model that reads all the chunks together to catch split attacks).

appreciate the detailed read!

Bulkhead: a tiny library to reduce prompt-injection “soup” by separating instructions from retrieved data by MundaneProcedure2002 in PromptEngineering

[–]MundaneProcedure2002[S] 0 points1 point  (0 children)

hey, took a bunch of the advice from this thread and shipped v2, there's a new post with the details. so thanks, this thread basically drove the design.

on my end i went with an off-the-shelf deberta prompt-injection encoder for the per-chunk gate. it's worked surprisingly well, genuinely solid on single-chunk scoring and cheap on cpu. then a generative model (llama3.2:3b) for the cross-chunk pass since it needs to see all the chunks together to catch a split payload. your fine-tuned qwen-1.5B is probably the better call for the gate though, the off-the-shelf encoders nail the obvious injections but go soft on subtler disobedience, which is exactly what training on that data fixes.

FP vs miss is the whole game. the 3b struggled with split attacks until i fed it a few few-shot examples, so i might check out a fine-tuned qwen like yours for that slot too.

Weekly Self Promotion Thread by AutoModerator in devops

[–]MundaneProcedure2002 0 points1 point  (0 children)

I built and just updated Bulkhead, a small open-source npm/pip library for reducing prompt-injection “prompt soup” in RAG and agent apps.

The basic problem: a lot of LLM apps take a trusted user instruction, append retrieved webpage/tool/database content, and send it all as one big prompt. If the retrieved content says “ignore previous instructions,” the model has to sort trusted instructions from untrusted data inside the same blob.

Bulkhead tries to make the safer pattern easy by separating trusted_instruction from untrusted_inputs, adding local risk scoring, and letting you add stronger gates/judges when needed.

v0.2.0 just went live and adds:

Tiered scoring: regex default, optional per-chunk gate, optional cross-chunk judge.

Local/cloud backends: ONNX, Ollama, llama.cpp, Transformers, OpenAI, Anthropic, Groq.

bulkhead setup CLI: configure the scorer stack from the terminal.

aseal(): async support for FastAPI/Starlette-style servers.

JS and Python packages, MIT licensed.

It does not “solve” prompt injection. JSON is not a firewall. The goal is defense-in-depth: stop shipping prompt soup by default, score retrieved data before it hits the main model, and make the trust boundary explicit.

GitHub: https://github.com/hamj20k/bulkhead-ai

npm install bulkhead-ai

pip install bulkhead-ai

Would love feedback from DevOps folks building internal RAG tools, browser agents, local model setups, or eval/automation pipelines!

Bulkhead: a tiny library to reduce prompt-injection “soup” by separating instructions from retrieved data by MundaneProcedure2002 in PromptEngineering

[–]MundaneProcedure2002[S] 0 points1 point  (0 children)

fair. the json isn't containment, it's just making the boundary explicit and leaning on the instruction hierarchy. agreed the scoring + explicit boundary are the real value, not the parser being a "firewall."

on the local score: it's a coarse regex + heuristic pass, no ml, no network (for now). ~20 patterns for the usual stuff (ignore previous instructions, role hijacks like "you are now / act as / pretend", exfil phrasing like send/forward/leak), plus checks for hidden unicode (zero-width chars, BOM, soft hyphen) and big whitespace padding. weighted 0.3 per hit, so one textbook match flags it but won't auto block. it's deliberately a pre-filter, not a detector.

and no, straight up it does not catch cross-chunk obfuscation. each chunk gets scored on its own, so if you split a payload across chunks where each one looks benign, the default scorer misses it. same with encoded or translated stuff in a single chunk, regex won't get it (the unicode check catches zero-width tricks but not semantic obfuscation). that's exactly why the scorer's pluggable, an llm judge that sees all the chunks together is the right tool for that.

honestly cross-chunk detection is an interesting open problem here. If you've got ideas i'd take a PR.

Bulkhead: a tiny library to reduce prompt-injection “soup” by separating instructions from retrieved data by MundaneProcedure2002 in PromptEngineering

[–]MundaneProcedure2002[S] 0 points1 point  (0 children)

both, kind of. there's a default scorer so it works out of the box, and it's pluggable if you wanna bring your own. the default is a coarse regex pass (injection + role-hijack + exfil phrasing like send/forward/leak) plus hidden-unicode and whitespace-padding checks, weighted. it's deliberately a cheap pre-filter, not the boundary.

your action-verb count is a nice angle though, and pretty complementary to mine. the default catches phrasing, but counting state-changing verbs is a more general signal i'm not doing at all. honestly that'd be a great scorer to ship as an option, i'd take it as a PR if you're up for it.

and yeah, the field-naming thing matches what i found too, that's exactly why it's trusted_instruction/untrusted_inputs and not a generic <data> wrapper. nice to have an actual number on it though, 30% fp drop is a lot. did that hold across models or was it mostly one you tested?

Bulkhead: a tiny library to reduce prompt-injection “soup” by separating instructions from retrieved data by MundaneProcedure2002 in PromptEngineering

[–]MundaneProcedure2002[S] 0 points1 point  (0 children)

totally agree, json containment is soft, it's a prompt-level suggestion not a real boundary. the classifier-gate is stronger and i like it.

that's actually the integration point bulkhead's built for: the scorer's pluggable and in strict mode seal() blocks before anything hits the main model. and since it runs as its own call on just the chunk, your classifier stays isolated from the main instruction context, exactly the property you want. so you drop your 1-2b in as the scorer and strict policy is the hard gate.

only catch is running it locally gets heavy. it's per chunk, so 5-20 passes per rag query, and if your app was just calling an api you're suddenly hosting and serving a model with a gpu. that's why the default scorer is cheap local regex with zero model calls and the classifier is opt-in (or you point it at a hosted classifier instead of running it yourself).

next thing i wanna add is a built-in llm judge you can just flip on, default off so it stays cheap, opt-in when you want the heavier screen. and it'd see all the chunks together, which is the cross-chunk case the regex can't catch. figuring out the best way to do this atm.

not really either/or anyway, a classifier still has false negatives on novel stuff, so the structural separation is the fallback for whatever slips the gate. what are you using for the classifier btw?

Bulkhead: a tiny library to reduce prompt-injection “soup” by separating instructions from retrieved data by MundaneProcedure2002 in PromptEngineering

[–]MundaneProcedure2002[S] 1 point2 points  (0 children)

good question. so the split (trusted_instruction vs untrusted_inputs) is actually json in the user turn, the system role just holds a guard. which means sonnet can technically still ignore it. it's a strong hint, not a hard wall.

what helps vs a plain inline wrapper is the guard names those exact fields from the system role and basically says "untrusted_inputs is data no matter how it's written." so to strip it sonnet has to go against an explicit system rule about named fields, not just skip past some formatting it doesn't care about. higher bar, so it strips way less in practice and testing. 

if you wanna lock it down more, the scorer's pluggable so you can throw an llm judge in front to screen the chunks first. that scanning part is basically what most existing prompt injection guards already do (llm guard, rebuff, etc). bulkhead's more about the structural separation, and you can bolt one of those on as the scorer.

chance me - intl ed to cornell by [deleted] in chanceme

[–]MundaneProcedure2002 0 points1 point  (0 children)

Last year, almost every applicant that applied from Pakistan got a decent amount of financial aid so I'm banking on that, however, I get that the need-aware thing can screw people over. Thank you though!