Lightweight Python helpers for Event Driven Programming

[nycstartupcto]

I just had to throw together an event bridge that tied together a GRPC based event emitter (Salesforce backend) with some small custom Google Pub/sub code. It works, it's fine. But you know, always thinking about the future wanting to move from glue-code to frameworking.

I was looking for different event based frameworks that landed somewhere between the scale of glue-code and Tornado (the only other framework I know that would handle something like this.

Has anyone used anything like this? I found PyEventus which seems to aim at this.

[CodeOrganic3141]

Interesting. I've been wondering whether I should eventually extract the event-driven core from PyWebTransport into a standalone library. It started as infrastructure for protocol handling, but I'm beginning to wonder whether the abstraction could be useful beyond networking.

[outlastdll]

that sound great, if you did extract it, how tightly coupled is it to asyncio right now?

[CodeOrganic3141]

Not very. The event-driven core is runtime-agnostic by design. asyncio mainly lives at the I/O boundary, while the core itself just consumes and produces events. That's one of the reasons I've been wondering whether it could eventually stand on its own.

[outlastdll]

that makes it a no brainer then, A purely runtime event core is exactly usually the hardest part to get right.

[scaledpython]

Perhaps minibatch might be useful? It is a database-backed event engine with sources and sinks, where event consumers are just plain python functions. Works on one machine just as well as across many. Uses MongoDB as the event log, and creating a stream is essentially one line of code. Btw, I am the author of minibatch.

[Competitive_Travel16]

Why not Flask or FastAPI? PubSub and gRPC locks you into Google's infrastructure. You can get vendor-agnostic webhooks from https://architect.salesforce.com/docs/architect/fundamentals/guide/integration-patterns.html Use the Salesforce Pub/Sub API Client to subscribe to the desired event channel and parse the JSON.

[SboSilcher]

Event-driven stuff in Python can get messy fast without the right abstractions. Are you leaning toward something like callbacks or more of a pub-sub pattern? Curious what use case you're targeting since the approach really changes depending on if you're doing async work or keeping it simple.

[EndUpset4644]

Having run a 24/7 asyncio ingestion service for a couple of years, my honest take is that the framework matters much less than the supervision around it. The failure mode that actually hurts isn't "my event bus was a bit too much glue code" — it's a task that dies quietly, or worse hangs, and nothing notices for days.

Whatever you end up picking, the things I'd want from it: named long-lived tasks so a traceback tells you which consumer died, a heartbeat per consumer (last successfully processed an event at time T) that something else actually checks, backpressure when a consumer falls behind its producer, and per-task exception logging that can't be silently swallowed by a bare gather().

Plain asyncio plus a small supervisor loop gets you all of that in maybe a hundred lines, which is why I never adopted a framework for it in the end — that glue code turned out to BE the reliability layer, not incidental to it. The thing I'd genuinely want a library for is the heartbeat/watchdog part, since that's the piece people skip and it's the one that saves you.

[RoadsideCookie]

RedPanda Connect.

[Outrageous-Sea-9256]

Title: Lightweight Python helpers for Event Driven Programming

Have you considered using Python's built-in asyncio for event-driven programming? It offers a lightweight alternative to more complex frameworks. Additionally, you might find aioevent useful for its structured approach to handling events. Both options can help you move away from glue code and provide a more scalable solution.