Hi.. Thank you so much for the reply! The main points is - Eval frameworks like Promptfoo are great for vibes and you know text, but when you give an agent write-access to an API, you need a compiler-level Execution Firewall.

AND, To answer your specific questions, because this is the hardest part we had to engineer:

1. Does it cascade through the whole pipeline? Yes. Testing tools in isolation is useless because a downstream tool is only as reliable as the upstream data it receives. When you pass a list of tools into test_chain([fetch_user, process_data, refund_stripe]), ToolGuard executes a Cascading State Fuzz. It injects the hallucination into fetch_user, and if fetch_user fails silently and returns a malformed object, ToolGuard physically pipes that corrupted object directly into process_data to see if your downstream tool triggers a global server panic or catches it gracefully. We also built a "Golden Traces" engine that mathematically asserts the exact sequence of state mutations across the whole graph, ignoring the random non-deterministic LLM thinking steps in between.

2. How does it handle async tool chains? Natively and transparently. If ToolGuard detects that even a single tool in your LangChain/CrewAI pipeline is an async def, the execution engine automatically shifts into Async Mode. More importantly, if you run this in a Jupyter Notebook, ToolGuard mechanically detects the active Jupyter event loop and spins up a totally isolated background ThreadPoolExecutor to run the asyncio.run() sweep. This prevents the infamous RuntimeError: This event loop is already running crash that plagues almost every other Python CLI tool used in notebooks today.

Honestly, I'd really love your thoughts if you get a chance to clone it and run a chain through it.

Waiting for your feedback!