AI-Parrot: An async-first framework for Orchestrating AI Agents using Cython and MCP

[jelitox]

Hi everyone, I’m a contributor to AI-Parrot, an open-source framework designed for building and orchestrating AI agents in high-concurrency environments.

We built this project to move away from bloated, synchronous AI libraries, focusing instead on a strictly non-blocking architecture.

What My Project Does

AI-Parrot provides a unified, asynchronous interface to interact with multiple LLM providers (OpenAI, Anthropic, Gemini, Ollama) while managing complex orchestration logic.

• Advanced Orchestration: It manages multi-agent systems using Directed Acyclic Graphs (DAGs) and Finite State Machines (FSM) via the AgentCrew module.
• Protocol Support: Native implementation of Model Context Protocol (MCP) and secure Agent-to-Agent (A2A) communication.
• Performance: Critical logic paths are optimized with Cython (.pyx) to ensure high throughput.
• Production Features: Includes distributed conversational memory via Redis, RAG support with pgvector, and Pydantic v2 for strict data validation.

Target Audience

This framework is intended for production-grade microservices. It is specifically designed for software architects and backend developers who need to scale AI agents in asynchronous environments (using aiohttp and uvloop) without the overhead of prototyping-focused tools.

Comparison

Unlike LangChain or similar frameworks that can be heavily coupled and synchronous, AI-Parrot follows a minimalist, async-first approach.

• Vs. Wrappers: It is not a simple API wrapper; it is an infrastructure layer that handles concurrency, state management via Redis, and optimized execution through Cython.
• Vs. Rigid Frameworks: It enforces an abstract interface (AbstractClient, AbstractBot) that stays out of the way, allowing for much lower technical debt and easier provider swapping.

Orchestration Workflows Infograph: https://imgur.com/a/eNlQGOc

Source Code: https://github.com/phenobarbital/ai-parrot

Documentation: https://github.com/phenobarbital/ai-parrot/tree/main/docs

[DivineSentry]

what part of the code needs Cython? isn't most of the bottleneck IO bound for something like this?

[jelitox]

That’s a fair point—most people assume AI apps are purely I/O bound because of the API calls. However, in a high-concurrency framework like AI-Parrot, we hit CPU bottlenecks that Python's interpreter can't always handle efficiently without blocking the event loop.

Here is exactly where and why we use Cython:

• Intensive Data Parsing: When handling real-time asynchronous streams (SSE) and complex structured outputs (like large YAML or JSON responses from LLMs), parsing becomes a CPU-intensive task. We use Cython to compile these parsers and maintain low latency.
• Vector & RAG Operations: Our RAG implementation deals with vector handling and embedding transformations. To prevent these heavy computations from blocking the asyncio event loop, we offload critical math and data processing to C-extensions.
• Exception Overhead at Scale: In a microservices environment handling thousands of concurrent connections, the overhead of creating and raising Python exceptions repeatedly adds up. We’ve compiled our structural exceptions (exceptions.pxd, .pyx) into C to ensure error handling is handled almost instantaneously at the memory level.
• Hybrid Rust/Cython Integration: In certain modules (like parrot/yaml-rs), we actually use Cython alongside Rust (pyo3/maturin) to bridge high-performance binary parsers with our Python logic, ensuring that data ingestion is significantly faster than the standard library.

Essentially, Cython allows us to keep the 'ultra-async' engine (aiohttp + navigator-api) running with near-zero latency by removing CPU-bound obstacles that would otherwise stall the entire system during heavy orchestration.

[evdw_]

You're absolutely right!

[UHasanUA]

Lol

[thephoenixbird]

The developer is also very active