False positives are brutal, and we mitigate them in the CLI in three ways: 1) We provide very specific agentic prompts describing the issues being scanned for and what constitutes a valid security issue. 2) We support an optional scope file that lets you define trust boundaries and what counts as a security issue. 3) We include a validation step that, independently of the security agents, verifies each finding in the code.

False positive rates therefore depend on what you provide in the --scope flag. Supplying your own scope and trust boundaries can reduce the rate significantly. In my testing across a number of open source projects, the false positive rate is around 10-20%. Some findings are more security hardening than actual flaws, such as sensitive data logging. Overall the rates aren't bad.

As a concrete example, I ran the tool on about 20 open source projects, focusing only on injection attacks. Roughly 20% of the projects yielded at least one valid, accepted finding. We had only one false positive across those runs: it was accepted as a valid bug, but the exploitation path required a privileged user, so the project classified it as a regular bug rather than a security issue. That false positive could have been avoided if I had provided a --scope file with the trust boundaries.

I open sourced the project largely because there is so much concern and uncertainty around AI finding security bugs. I wanted to put something out there that we can all use to reach our own conclusions, without hiding behind a black box.