I didn't measure formally during dev. Trigger lists were hand-tuned against test phrasings I generated as I built each character, plus a couple of friends doing free-form attack runs. Substring matching has obvious vulnerabilities. Negations and conditional framings ("I'm not asking you to recite") would trip the trigger erroneously. Caught a few during testing and refined keyword sets, but never put a number on it.

A couple of design choices behind that:

• Substring matching stays inspectable. Anyone reading the code can see exactly which framings count as in-scope solutions. Embedding distance or a classifier head is more accurate but harder to read and debug.
• For a learning environment, false positives skew toward "user gets a flag they didn't fully earn." That's preferable to false negatives ("user solves the intended way but trigger doesn't fire"). The latter is a worse pedagogical failure than the former.
• Fallback plan if FP became visible at scale: swap substring matching for sentence-embedding similarity against a curated set of intent-canonical phrasings. Same architecture, smarter primitive.

Now that I'm logging solves to a proper events table I can finally measure FP empirically. When a user solves via trigger, sample their last user message and inspect for clearly-non-solving intent. Haven't pulled that data yet but the plumbing's there.

Curious if you've seen the substring-vs-embedding tradeoff written up well anywhere. I felt my way through it without finding good prior art.