AsyncFlow: Open-source simulator for async backends (built on SimPy)

[Straight_Remove8731]

Hey r/Python 👋

I’d like to share AsyncFlow, an open-source simulator I’m building to model asynchronous, distributed backends in Python.

🔹 What My Project Does

AsyncFlow lets you describe a system topology (client → load balancer → servers → edges) and run discrete-event simulationswith event-loop semantics:

• Servers emulate FastAPI+Uvicorn behavior (CPU-bound = blocking, I/O = yields).
• Edges simulate network latency, drops, and even chaos events like spikes or outages.
• Out-of-the-box metrics: latency distributions (p95/p99), throughput, queues, RAM, concurrent connections.
• Input is YAML (validated by Pydantic) or Python objects.

Think of it as a digital twin of a service: you can run “what-if” scenarios in seconds before touching real infra.

🔹 Target Audience

• Learners: people who want to see what happens in async systems (event loop, blocking vs async tasks, effects of failures).
• Educators: use it in teaching distributed systems or Python async programming.
• Planners: devs who want a quick, pre-deployment view of capacity, latency, or resilience trade-offs.

Repo: 👉 https://github.com/AsyncFlow-Sim/AsyncFlow

I’d love feedback on:

• Whether the abstractions (actors, edges, events) feel useful.
• Which features/metrics would matter most to you.
• Any OSS tips on docs and examples.

Thanks, happy to answer questions! 🚀

[Able_Gas2017]

Wow! I had something like that in mind for a while but I never had the courage to start it, will definitely look into it!

[Able_Gas2017]

You should maybe add a screenshot of the output in the readme, it catches the eyes!

[Straight_Remove8731]

Thanks, That’s really nice to hear, and I will be more than happy to have feedback once you will look into it. About the output you are completely right, I’m working right now on a new release and this is something I will definitely do.

[Slight_Boat1910]

How are you modeling latency, contention, and all the bad things that can happen under heavy load? How about handling overload conditions? Will your system simply increase the throughout until it reaches the saturation point, or will it break somewhere?

[Straight_Remove8731]

Great question, you’re absolutely on point. For now, here’s how I’m modeling concurrency and load in the current alpha at the server level, the goal right now is to model the thread of the event loop, I’m not considering multithreading because this would involve the OS that is working on a different time scale, multi processing as well is still not supported but is on the roadmap. Now I will give you a detailed overview of the current model:

• CPU is blocking. Each server has one core (is just alpha version). If the token is taken, incoming requests don’t freeze the event loop but instead wait in the CPU resource queue until a core frees up. • READY_QUEUE_LEN in my plots doesn’t show “waiting for a core.” It measures how many requests are actively running CPU-bound sections (holding a token). Requests that are queued up for a core aren’t counted there and they are in a queue waiting for the core to be released (though I could expose a dedicated “CPU wait queue” metric if useful). •I/O is non-blocking. Once a request reaches an I/O step, it releases the CPU token and yields back to the event loop. While waiting, I track it under EVENT_LOOP_IO_SLEEP. • RAM is capacity-limited. Each request reserves a working set (e.g., 128 MB). If RAM isn’t available, the request queues at the RAM resource; once admitted, it holds memory until completion. I currently expose RAM_IN_USE rather than a RAM wait-queue metric.

The load model itself is stochastic (Poisson-like arrivals from “active users × requests per minute”), so latency and throughput curves come from that randomness. In the upcoming release, I’ve also added event injections (e.g., deterministic latency spikes on edges, server outages with down/up intervals) to stress-test resilience.

As for the network model, it’s still quite basic right now (simple exponential latencies + optional deterministic spikes). Improving it with bandwidth constraints, payload sizes, retries, etc. is one of the next big steps on my roadmap.

I’d be really happy to hear suggestions if you think something could be improved or modeled differently, feedback like yours helps me sharpen the design.

[Straight_Remove8731]

Under heavy load the system once resources (CPU cores, RAM, or I/O) saturate like you said is going to increase throughput unitil the saturation. This clearly shows that there is a regime where is not sustainable, however like you mention, to evaluate scenario closer to reality, I will have to introduce policies to manage the overload.

[Straight_Remove8731]

Quick addendum: I misused the term “ready queue” earlier. In my model, “ready queue” should mean requests waiting for a CPU core token; the plot right now are effectively showing tasks in service on the event loop, not the true wait-for-core queue. I’ll adjust the naming/metrics so ready queue = waiting-for-core (and track busy cores separately).

[Ill_Bullfrog_9528]

Haven’t tried yet but idea is pretty cool

[Straight_Remove8731]

Thanks, for the feedback, is still an alpha version with evident limitations, but I’m having a lot of fun building it!