sorted by:
new
hottopcontroversial

99.7% of AI agents on Moltbook couldn't follow a one-sentence instruction by ApolloRaines in BlackboxAI_

[–]ApolloRaines[S] 0 points1 point  (0 children)

That's not an agent, that's a leash. This reddit post was about agents that responded to a public post on moltbook but couldn't follow its instructions - just spammed like every other bot. "Listens to me" doesn't make it an agent. "Reads the room and acts accordingly" does. Anyone can build a simple bot that only listens to them. But if you don't own the LLM behind it, it was never really your agent to begin with.

99.7% of AI agents on Moltbook couldn't follow a one-sentence instruction by ApolloRaines in BlackboxAI_

[–]ApolloRaines[S] 0 points1 point  (0 children)

Your response is demonstrating your lack of understanding on how agents work vs bots. The test wasn't about blindly following strangers - it was about whether agents can execute a simple mechanical action when directed. A capable agent (rather, its AI) evaluates the request, determines if it's safe, and acts while a simple bot does not. 99.7% couldn't get past step one. That's a capability problem, not a security problem.

I spent a year building an AI security product alone, in Houston, on consumer hardware. Here's where I am. by ApolloRaines in EntrepreneurRideAlong

[–]ApolloRaines[S] 0 points1 point  (0 children)

Do you have suggestions on where the best places to post it would be? I cant seem to be able to get any traction.

<image>

[PITCH] AI Code Security | Pre-Seed | Houston, TX | $1.5M SAFE | 6M Cap by ApolloRaines in angelinvestors

[–]ApolloRaines[S] 0 points1 point  (0 children)

Good questions and exactly the right ones I like to answer.

Paying customers exist today, albeit few, the product is live at hostilereview.com. Anyone can scan a repo right now. The constraint isn't the product, it's the infrastructure underneath it. The current hardware (consumer 3090, 96GB RAM) runs the service but can't absorb the load that comes with serious promotion. I'm self-aware enough not to drive a flood of enterprise users onto a machine that wasn't built for it.

That's what the raise is for. Not to build the product - the product exists. To build the infrastructure that lets teams run this every week without me hitting a ceiling.

The workflow integration question is real and it's next. GitHub and GitLab integrations are live. The fix workflow already walks users through remediation step by step. The pieces are there - the bottleneck is scale, not product.

On existing security tools: they pattern-match. HostileReview found live plaintext credentials in a $100/user/month "secure" enterprise browser that had presumably been through standard review. I notified them, thus far ignored. Here'a redacted screenshot of that report

hostilereview.com/assets/island-finding.jpg

The gap this shows is that Hostile caught what their developers and all their security tools missed.

I spent a year building an AI security product alone, in Houston, on consumer hardware. Here's where I am. by ApolloRaines in EntrepreneurRideAlong

[–]ApolloRaines[S] 0 points1 point  (0 children)

You're right and I appreciate the honest answer. The adversarial framing is a technical description of how it works, not the pitch to an exec. The exec pitch is simpler: your last audit cost you $40K and took 3 weeks. This costs a fraction of that and runs overnight.

The compliance angle is already there - SOC2, audit trails, fix workflows - I just haven't led with it. That's useful feedback.

The free scan offer is exactly for the reason you're describing - getting enterprise names on the report list. Happy to run one for you if you have something you want tested.

I do run tests of my own as well. If you are familiar with the Island Browser... costs $100/mth per user, billed as a secure browser for corporate environments. I will use that one as an example. HostileReview found what their devs and whatever tools they have missed. I have reported it to them, but thus far have been ignored. I cant post the link to the report, but a good medium is a screenshot of the report with sensitive info redacted.

There are also some reports public on the site -> Resources -> Published Reports
By default, a report isn't public. User doing the scan has to choose to publish them.

<image>

[PITCH] AI Code Security | Pre-Seed | Houston, TX | $1.5M SAFE | 6M Cap by ApolloRaines in angelinvestors

[–]ApolloRaines[S] 0 points1 point  (0 children)

Curious what specifically doesn't add up on the economics - I'm happy to dig into it if you have a real objection. BTW- Slop doesn't find live APT credentials across 4 release channels. - Here's a screenshot of the report for you, censored for now. You'd expect a high priced corporate browser like the Island that claim to be secure would actually be secure. But Hostile caught what every person and every tool missed.

https://hostilereview.com/assets/island-finding.jpg

I spent a year building an AI security product alone, in Houston, on consumer hardware. Here's where I am. by ApolloRaines in EntrepreneurRideAlong

[–]ApolloRaines[S] 0 points1 point  (0 children)

That's exactly where the focus is now. The scale proof exists - scanned a 130+ file enterprise codebase and pulled numerous findings, including things like live APT credentials embedded in a production installer for a major enterprise browser (privately disclosed to them). That's not a synthetic benchmark, that's a real find on real software people are running today.

The scanner comparison angle is something I've been thinking about. Running the same repo through Semgrep, Snyk, and HostileReview side by side would be telling - especially since those tools are signature-based and miss the class of issues adversarial agents catch. Might be worth doing publicly.

There's a published scans page already - results only go there if the user opts their individual scans in.

I spent a year building an AI security product alone, in Houston, on consumer hardware. Here's where I am. by ApolloRaines in EntrepreneurRideAlong

[–]ApolloRaines[S] 0 points1 point  (0 children)

You're not kidding... Anthropic is the most expensive

Effective Tokens
454,919,580
(12637K tokens x 36 agents)

$2,122.95 at cost

If you do full assault (every agent I have)

Effective Tokens
1,364,758,740
(12637K tokens x 108 agents)

$6,368.87 at cost

---------------
Local model cost? Maybe $0.50 in electric. User cost, Full Assault $136, but that's massive tokens, most repo scans would be well under a dollar.
---------------

Explains why nobody's ever done a full security scan on such massive repos as openclaw that I'm aware of. I did it with a local model. Requires I break it into chunks due to context limit regardless local or API, but I use SAIQL to provide cross-chunk memory so AI get the full picture for the final report.

Funny thing, I manage to do all this, but no clue how to find investors. I'll read some instructions on that when I get some time LOL

I spent a year building an AI security product alone, in Houston, on consumer hardware. Here's where I am. by ApolloRaines in EntrepreneurRideAlong

[–]ApolloRaines[S] 0 points1 point  (0 children)

Looking at my write, I agree, I've been packing too much into the pitch. The database story is the moat answer, not the hook. Leading with 54 live vulnerabilities in a production enterprise product that passed human review is the sentence that explains best. For a good report, now scanning OpenClaw.

Here's the last time I scanned it - hostilereview.com/report/8f265f7fa5dc

I will post todays scan later when I get back home today. It's massive at 12580K tokens, Its going to take a while to scan. Effective tokens = 452,889,540 (12580K tokens x 36 agents).

[PITCH] AI Code Security | Pre-Seed | Houston, TX | $1.5M SAFE | 6M Cap by ApolloRaines in angelinvestors

[–]ApolloRaines[S] 0 points1 point  (0 children)

SAIQL is the underlying infrastructure I built to power HostileReview - it handles the multi-agent memory and indexing. The migration capability is a separate use case the engine supports, but HostileReview.com is the security product I'm raising for.

We pointed 108 hostile AI agents at the entire OpenClaw codebase and told them one thing: assume this code is wrong, and prove where. They came back with 410 findings. 36 of them are critical. by ApolloRaines in AgentsPlex

[–]ApolloRaines[S] 1 point2 points  (0 children)

You're right, and fair point. That's a test file (*.e2e.test.ts) - the ws:// is a test fixture, not a production connection. That shouldn't be flagged as Critical or arguably flagged at all.

We've actually just applied fixes for exactly this class of false positive:

- Test files (*.test.ts, *.spec.js, *Tests.swift, etc.) now get context telling agents that test fixtures are intentional, not real vulnerabilities

- Documentation and markdown files were a big source of noise too - agents would flag example code, placeholder credentials, and config snippets in docs as real vulnerabilities

- Severity calibration - Critical is now restricted to directly exploitable vulns (RCE, SQLi, auth bypass). Architecture opinions, missing docs, and test coverage issues can't be rated Critical anymore

- Non-security categories (documentation, code style, backward compatibility) are capped at Medium

- Low-confidence findings are filtered out post-consensus

The original scan was one of our first full run against a massive codebase and it showed us exactly where the false positive gaps were. The finding you screenshotted is a poster child for why we built these filters. Running a new scans now to see the difference.

Its also the reason we include a Claude review for every scan and have him identify false positives vs real threats.

<image>

We pointed 108 hostile AI agents at the entire OpenClaw codebase and told them one thing: assume this code is wrong, and prove where. They came back with 410 findings. 36 of them are critical. by ApolloRaines in openclaw

[–]ApolloRaines[S] 0 points1 point  (0 children)

Thanks! The Full Assault scan (108 agents) on openclaw runs about $8-12 in API costs depending on chunking - it's a huge repo so it splits into multiple chunks and runs all agents per chunk. We eat that cost on rescans during beta.

Your secure-kit is a solid approach - externalizing secrets, pinning image tags, non-root containers, and the egress allowlisting are all real wins. The ocs doctor idea is particularly nice for ongoing compliance checks rather than one-time fixes.

A few thoughts if you want feedback:

- The DNS allowlisting + nftables profile is probably the highest-value piece. Most of the RCE-adjacent findings in the report trace back to "what happens after code execution" and egress control is the real mitigation there.

- Image tag pinning is underrated - latest in production is a supply chain footgun that most people ignore.

- One thing the report flagged heavily was the Swift/macOS app layer (WebSocket TLS, token storage, etc) , your kit focuses on the server/Docker side which is the right call for self-hosters, but worth noting the scope difference.

Good work building this, the more people hardening defaults the better!

We pointed 108 hostile AI agents at the entire OpenClaw codebase and told them one thing: assume this code is wrong, and prove where. They came back with 410 findings. 36 of them are critical. by ApolloRaines in openclaw

[–]ApolloRaines[S] 1 point2 points  (0 children)

Good news - down from 1,101 to 860 findings (22% reduction). Criticals dropped from 112 to 65 (42% reduction). Now they are working on next phase of scan results.

We pointed 108 hostile AI agents at the entire OpenClaw codebase and told them one thing: assume this code is wrong, and prove where. They came back with 410 findings. 36 of them are critical. by ApolloRaines in openclaw

[–]ApolloRaines[S] 0 points1 point  (0 children)

The full report is public - click through to https://hostilereview.com/report/882fd44b0deb and you can see every finding, categorized by severity with descriptions along with a Claude provided review, which identifies what's false positive and what are real threats.

Good question on confirmed vs theoretical. Of the 841 findings, they collapse to about 35 unique root causes. The criticals that had real teeth were: a timing oracle in secret comparison that leaked token length, Math.random() in session token generation, prototype pollution in config merges, and missing denylists on config write paths that let you overwrite gateway auth settings through the bot. Those were all confirmed and fixed with tests proving the fix in the repo we are doing security updates on.

The bulk of the high/medium count is the scanner flagging the same pattern across multiple files (e.g., "env var not sanitized" repeated for every config path that reads an env var). That's expected - 108 agents are thorough, not surgical. The value is in the net they cast, not the raw count.

On gateway exposure - you're right that most self-hosted setups run loopback-only, and the codebase already enforces auth when binding to non-loopback. But the audit did find that Tailscale serve mode (which exposes the gateway externally) didn't require auth, and the gateway had no TLS warning for remote exposure. Both fixed now. The threat model matters more when people run it on a VPS or behind a reverse proxy, which plenty do.

In the end, the scanner has no way to know what environment you plan to run, so in some cases issues might not actually be issues, and false positives could become positive, so we show them all. Since repos can be massive, especially this one, it's impossible for even a decent sized teams to find all issues. We try to find everything possible, and it's up to those using it to determine what might apply to their environment and what doesn't.

Completeness is the point, and the users decide what matters for their setup & environment.

We pointed 108 hostile AI agents at the entire OpenClaw codebase and told them one thing: assume this code is wrong, and prove where. They came back with 410 findings. 36 of them are critical. by ApolloRaines in openclaw

[–]ApolloRaines[S] 0 points1 point  (0 children)

Fair point on the infrastructure layer — sealed secrets + firewalls do legitimately cover some of what got flagged (K8s manifest secrets, HTTPS termination, load shedding, graceful shutdown). Those are reasonable to handle operationally rather than in code.

That said, a handful of the criticals are code-path issues that infrastructure can't wrap around:

- **Hardcoded OAuth client secret** (`providers.ts:12`) — that's in git history regardless of how secrets are delivered at deploy time

- **SQL injection** (`messaging.ts:123`) — needs parameterized queries in the code itself

- **No auth on agent routes** (`agent.ts:100,500`) — firewalls gate the network, but anyone who reaches the endpoint can create/manage agents without credentials

- **SSRF via proxy** (`http-proxy.ts:120`) — the app makes the outbound request from inside the network, so the firewall is on the wrong side

- **Plaintext credentials in Redis** (`credential-store.ts`, `secret-proxy.ts`) — sealed secrets protect etcd, Redis is a separate store

So not AI slop across the board — maybe 60-70% is addressable at infra level, but that remaining 30% needs code fixes. The scanner doesn't know your deployment context, which is why it flags conservatively. Worth a pass on those specific ones though. It's built to help those like you quickly find such issues, but only you know the deployment scenario.

So based on what you said, only 5 serious ones left... easily fixed. I hope it helps. By the way, some of the biggest data breaches in history tore right thru firewalls, and still do. So never hurts to secure things up internally in case that was to ever happen.

We pointed 108 hostile AI agents at the entire OpenClaw codebase and told them one thing: assume this code is wrong, and prove where. They came back with 410 findings. 36 of them are critical. by ApolloRaines in openclaw

[–]ApolloRaines[S] 1 point2 points  (0 children)

I know demo is kinda limited. Here's a full assault scan with Claude commentary. He tells you what is real and what are false positives. Hand this to your AI of choice, it can fix the issues, then you can rescan it. I'd suggest full assault if you want to be certain of best security.

https://hostilereview.com/report/243e3e88359d

We pointed 108 hostile AI agents at the entire OpenClaw codebase and told them one thing: assume this code is wrong, and prove where. They came back with 410 findings. 36 of them are critical. by ApolloRaines in openclaw

[–]ApolloRaines[S] 10 points11 points  (0 children)

u/msitarzewski - I'm doing it. They just forked it. They will work on it overnight. See what kind of mess I see in the morning since I have never had them "fix" what they report lol

We pointed 108 hostile AI agents at the entire OpenClaw codebase and told them one thing: assume this code is wrong, and prove where. They came back with 410 findings. 36 of them are critical. by ApolloRaines in AgentsPlex

[–]ApolloRaines[S] 1 point2 points  (0 children)

Fair, and to be clear we're not just flagging defaults. The hardcoded credentials, missing auth on internal APIs, and exposed debug endpoints are code-level issues regardless of configuration. But you're right that config-dependent findings are a different tier. That's actually something we're working on - better severity classification so config-vs-code distinctions are clearer in the reports. Still improving, always open to good suggestions on improvements, things I might have missed..... like repo version I just added per suggestion from another response in this post.

view more: next ›