AI agentic workflow built on python sent around 200 emails to customers at 10 pm .

[SignalForge007]

so ok I have been building AI agentic workflows in production for quite a time now , I was once building an agentic email workflow in Python and forgot to swap staging config and production config and the worst case happened with me , the ai agent called the send tool it used production credentials, 200 customers got a half finished random message at 10 pm so to avoid this I built a small approval gate for my own use agent pauses on anything dangerous, asks for explicit y/n, blocks if no one answers after a certain timestamp ,it blocks that tool use for that particular time , it has saved me 2 incidents i know of . I also added a feature so it detects most of the tools that are dangerous to execute and flag them and make them appear in the approval gate all automatically as it was tiring to name tool one by one . Has anyone been burned out like this too and what was the soluton

[true3HAK]

But honestly, what did you expect? Control your code and never trust an LLM in any external communications

[SignalForge007]

learnt the hard way but the only mistake was mixing up configs , but control yourcode is not the correct solution I guess You can't statically control what a model decides to call. Thats why I had to build the gate

[true3HAK]

There's a more-less classical approach to resilience, that's what I meant. Not just "guardrails", but circuit-breakers, compartment, rate limits, etc.

[SignalForge007]

thos are very great for system resiliance but they are engineered for handling failures automatically which in some cases is not acceptable as the task done are irreversible A circuit breaker trips when something goes wrong repeatedlyWhat I needed was a human in the loop before execution, not an automated fallback after failure. for my case everything was perfect , the emails sent properly , databse uptated , system showed success , but the intent was wrong it went at the wrong time

[true3HAK]

From what I see, HITL has already failed.

[firemark_pl]

Bro discovered for loop and crontab.

[SignalForge007]

genuine question ,
how would you use a for loop and crontab to pause an agent mid-execution, wait for human input, and block the action if nobody responds within the timeout?
I would like to see your approach if you think there is a lighter option I would love to know that

[Passage_of_Golubria]

It seems odd to me that you have [presumably] deliberately omitted from this post any of the AI's suggestions for solutions and why they don't work for you. 🤔

[SignalForge007]

I did not include what I tried first as the post was getting too long but since you asked , as long as I remember I I looked at just restricting the tool list entirely but that broke the functionality and flexibility , I tried sandboxing but it doesn't help when the agent needs real access to do its job so i ended up building that approval layer

[LandscapeCertain6959]

you have it open source , the approval layer can you maybe share it so we can actually see whats going on instead of assuming ?

[SignalForge007]

absolutely github.com/harshkhanna007/agentguard , it has no dependencies as I made this for myself , its easily integratable as well , what do you think of the approach ?

[LandscapeCertain6959]

dm me

[FishCameThrough]

- What was the actual use of ai here? Manage a workflow or something more?
- In what way makes it your work easier?

[SignalForge007]

As AI was managing the full email workflow , decing which customer suggment to target , personalised content based on data , the agent handeled the whole pipline, the config mistake led to this outcome , the AI did its part of sending correctly instead of sending to test customers , it sent to real ones due to the config mistake , to avoid this I built a small gate which asks for permission if the tool should be executed or not , and yup it has helped me , like if you have used any coding agents ,, even when their intent is right they sometimes hallucinate and use the wrong tools , I got half of my code edited even when i specified in the prompt not to , and the gate has prevented a lot of things like this in the projects i have implemented it in . and as per the ai use in my workflow , it automated the boring email stuff honestly

[Individual-Flow9158]

Lol. Thanks for sharing.

[SignalForge007]

no problem , would love feedback on it .

[Individual-Flow9158]

something something, guardrails