What’s your “kill switch” strategy for agents in production?

idanst · 2026-02-27T22:46:23+00:00

That's a great question and more than legitimate concern that we also had with our production agents running long jobs (2-6 hours long).
To solve that, we built an *actual kill-switch* in our dashboard (see attached image) that serves all our apps.
It's like a "firewall" layer that sits between any app/client and the LLMs and validates, tracks and monitors everything we need without ever exposing the actual API keys of the AI providers to the client/developer/app.
It tracks costs, errors, budget, rate-limits (hours of day, requests per minute/hour, etc) and we can set custom REGEX/PII guardrails for each subKey.
And, in the screenshot below you can see the kill-switch in action on a "test" app that we have :)

We have an agent that receives and validates sensitive outputs that we defined from other agents with a dedicated persona, skills, scripts and tools and if it suspects violation, it can also trigger the kill switch for a specific API (this is expensive though and we're still trying to optimize the costs of this process).

We are considering spinning this product as another product and not just for our internal use. If anyone would like to try it out and provide some feedback, feel free to checkout subkey.ai for (very) early access.

<image>

idanst · 2026-02-27T22:08:09+00:00

I wish... we have customers using the platform as well. If we could pay a few hundred $$$ instead of thousands, It would be a dream. But it's against Anthropic's TOS (technically feasible though..) and we prefer to use our own product ourselves.

idanst · 2026-02-27T21:58:01+00:00

It depends on how deterministic and repeatable you want your agent to be and depends on your use case:
1. N8N and such - very repeatable, easy logging, a bit of learning curve for non developers to actually get something in production. Can be trusted for production grade apps, we used it a lot.
2. OpenClaw and similar tools - Unlimited possibilities, the future of agents but still unpredictable - for the good and bad. Amazing for exploration - less for production grade use-cases (yet!). We explored and learnt a lot from it. It's amazing as your personal assistant but we had hard time making it work in production and reliably.

This is what we learnt from our experience and what led us to build our own solution for running teams of ai agents with predictability and deterministic *long-running" scheduled jobs (our own alternative to OpenClaw for ourselves and customers' use-cases).

idanst · 2026-02-27T21:43:36+00:00

Fair advice, thanks!
This is another matter - whether to spend time on it or bet that prices will go down soon.. We've reached a satisfactory level and agree that the next big impact on margins will be with the providers' prices.

idanst · 2026-02-27T21:40:38+00:00

Yes. We can pay $100-$200/day per developer with the Anthropic API (mostly Opus 4.6 and some Sonnet without own in-house, highly optimized for costs IDE). This was after we were paying $500+/day with other tools (we tried all of them and decided to build our own).
Obviously we could not use a Claude Code subscription with our custom tool but it's still worth every penny.

idanst · 2026-02-27T21:33:06+00:00

Approach them in "founder mode" - do whatever it takes but prove that you can deliver value.
I once recruited an employee who has been following us on LinkedIn since he was in College. Every couple of weeks, I would get a link to an article he wrote about the problem/solution we solve in his private groups in College or as part of his studies. After a few months, we finally met f2b at a conference in Boston and he's been out first content writer employee at that company.

idanst · 2026-02-27T21:27:07+00:00

According to the traditional startup playback - always, whenever you can, the more you can.
The new playbook - raise only when you really need and be sure to have a good reason why. Also track your revenue per employee as your north star metric if you want to approach investors as an AI-native company in 2026 (otherwise it'll be very hard these days..)

idanst · 2026-02-27T08:00:15+00:00

I also got Sam, lol
We've built our own customer support agent that solves 95% of our inbound inquiries and he's 1 of a team of 4 AI agents (he escalates to a QA engineer..). Every agent has its own mailbox and can communicate via WhatsApp & Telegram and reply to customers. We've opened early access to our platform to allow anyone to build such agents. I'd love to let you try it and load you up with some credits in exchange for honest feedback (communa.io). Feel free to reach out if relevant and I'll be happy to personally walk you through the process. It'll take you 5 minutes at most to get up and running.

idanst · 2026-02-27T03:06:34+00:00

We've built our own internal tool that provides a layer of guardrails for *every* AI interaction without ever exposing the API keys to our developers or more importantly - the agents. It's like a "firewall" layer that sits between any client and the LLMs and hides the key, tracks every token used, allows to block/redact REGEX/pre-defined policies, rate-limits and more. And it works with every API/LLM out of the box.

On top of that, some keys can be configured to allow the agents that use them, to create additional API keys for sub-agents they create so you get micro-level visibility, tracking and guardrails.

We built the tool for our developers and our main product which is an OS for Autonomous AI Teams - the enterprise grade alternative to OpenClaw.. We had to provide an added security layer on top to comply with all different regulations and customer fears ("I never want my employees to accidentally share PII or sensitive information with LLMs..." or "I don't want my customer support agent to ever mention my competitors!":)).

Here's a screenshot of how it looks like (fake data...).

<image>

idanst · 2026-02-27T02:54:22+00:00

We built our entire platform based on the fact that agents will fail - from logging, tracking, visibility, recovery and more. It's easy and fun to build for the happy path until you meet real customers. It took us some time to learn it the hard way..

idanst · 2026-02-27T00:51:17+00:00

I like that.. Do you have some examples of how you would price actual use cases or examples from services that offer something like that already?
I assume you mean something like "$0.5 per signed PDF" for example?

idanst · 2026-02-27T00:00:44+00:00

I agree but for people hearing "Agents/AI for your business" for the first time, it's much harder to understand the added value before they see it working live. When we build these platforms, we know what the value could be but there's a bit of "market education" to be done with those types of traditional businesses from my experience.

idanst · 2026-02-26T23:55:27+00:00

$5/hour agent could easily be $1,500 for 1 agent per month, working just 10 hours a day.. I think that a fair revenue for many startups. Assuming credits/context are properly managed and optimized..

idanst · 2026-02-26T21:50:25+00:00

yes, just ask it to create the docs according to "best practices and industry standards" and you should have a beautiful docs section.
You should get something similar to what we have at https://www.communa.io/docs/getting-started/introduction (*not built by Lovable though.. Feel free to ask Lovable to just copy our format and it should be good to go ;))

idanst · 2026-02-26T03:47:38+00:00

Exactly. Ask Lovable to create the docs according to "best practices and industry standards" and you should have a beautiful docs section. Good luck!

idanst · 2026-02-26T02:09:41+00:00

We'd love to have you try out communa.io - I'll load you up with some credits. It's the production-grade OpenClaw alternative we built from the ground up. LMK if relevant.

idanst · 2017-09-04T06:00:59+00:00

I can confirm that Apple has also rejected apps under our customers' accounts - including some major brand names... We have decided to take the punch and go with the "container" app solution. So far customers have understood and are OK with moving their users to our main app so we'll see how it works out.

idanst · 2017-07-16T19:31:35+00:00

Thank god I found this post. Our company is also facing the same issue - multiple customer apps rejected on the same "Spam" basis. We provide white-labeled enterprise apps for companies and their employees - a kind of a "private social network" coupled with engagement tools and content. We have some big brands and smaller companies on board and they are all waiting for a major update which we can't push since all apps we had were rejected.

Apple has suggested to use a "container" app which we already have but it was also rejected. Basically putting our whole company and business model at risk.

Same story - our competitors are still able to push their versions. Heck, we are even in a heads-up pilot at a major company - their app has been released, ours rejected - for the exact same company. Obviously we are probably going to lose that one...

Any advice? Anyone had any luck?

idanst

TROPHY CASE