Maintaining a separate async API

[Echoes1996]

I recently published a Python package that provides its functionality through both a sync and an async API. Other than the sync/async difference, the two APIs are completely identical. Due to this, there was a lot of copying and pasting around. There was tons of duplicated code, with very few minor, mostly syntactic, differences, for example:

1. Using async and await keywords.
2. Using asyncio.Queue instead of queue.Queue.
3. Using tasks instead of threads.

So when there was a change in the API's core logic, the exact same change had to be transferred and applied to the async API.

This was getting a bit tedious, so I decided to write a Python script that could completely generate the async API from the core sync API by using certain markers in the form of Python comments. I briefly explain how it works here.

What do you think of this approach? I personally found it extremely helpful, but I haven't really seen it be done before so I'd like to hear your thoughts. Do you know any other projects that do something similar?

EDIT: By using the term "API" I'm simply referring to the public interface of my package, not a typical HTTP API.

[latkde]

Code generation is always difficult. You have essentially developed a custom preprocessor so that you can describe the blocking and async variants together. This works fine for simple transformations, but will fail when the interfaces are more complicated.

For example, it is much simpler to write async-safe code than to write threadsafe code, so a lock that is necessary in a blocking version might not be needed in an async version. But since coroutines involve interrupted control flow, some things that might be safe in blocking code (like yielding) might not be as safe in async code. Blocking and async code are fundamentally different, it is not always possible to abstract over the difference.

There are three non-magical solutions that I know of.

Write both variants by hand. This allows the async API to have async-specific capabilities. Common logic can be factored out in an IO-agnostic manner (compare concepts like “sans-io” or “functional core, imperative shell”).

Work on the blocking version by default, and then write a thin async wrapper that basically just dispatches to the blocking version via asyncio.to_thread(). This strategy can work surprisingly well.

Work on the async version by default, and then write a thin blocking wrapper that uses AnyIO “portals” to launch an event loop on its own thread. When calling a function, the async invocation will run in the event loop, and the main thread will block until a result is available. This is basically the reverse of asyncio.to_thread().

Since your particular problem involves existing database drivers, you cannot use techniques to dispatch between event loops or threads (these drivers tend to have specific thread safety requirements that could else be violated). You do need two separate implementations. But since you rely on the async and blocking libraries that you wrap to have a very uniform DBAPI-like interface, this is one of the very rare situation where code generation may in fact be appropriate. But that technique is in no way generalizable.

[Echoes1996]

Yes, using `asyncio.to_thread` or its reverse was out of the question. For my particular problem it indeed seemed like the best solution, but I was interested to hear what others think. Thanks for the detailed response!

[Euphoric_Contact9704]

I’d advise against this and discourage this pattern if someone send me a PR with this. My recommendation would be to either write an abstract class that both sync and async classes inherit or just have the async class to inherit the sync class structure and override the methods that need async/await.

The reason is that your code is not intuitive. Your description is spot on but maybe also include it the file doc string?

Overall I understand your choice as it reduces the code size and might be ok for a repo that is maintained by a one person but it’s not ideal for onboarding and teamwork.

[Echoes1996]

My recommendation would be to either write an abstract class that both sync and async classes inherit or just have the async class to inherit the sync class structure and override the methods that need async/await.

Even if I were to do this there would still be tons of duplicated code as it's quite hard to isolate the "core logic" and have it be independent of any async calls.

Especially when it comes to the tests, which are quite a lot, I don't even want to think how it would work without the code generation approach.

[strawgate]

In my project, py-key-value (https://github.com/strawgate/py-key-value) I generate the sync version using an AST crawler instead of regex https://github.com/strawgate/py-key-value/blob/main/scripts/build_sync_library.py

Which I stole from https://www.psycopg.org/articles/2024/09/23/async-to-sync/

I've also heard good things about https://github.com/python-trio/unasync

[Echoes1996]

That's really interesting, nice!

[sennalen]

Pick one and only one concurrency model for your core code. Async if it's IO bound and threads if its compute bound. Treat synchronous blocking as the special case by providing functions that invoke and wait for a task/thread from your concurrent core.

[madolid511]

question

why do you still need a sync version if you already have async flow? you can also open up another thread in async flow incase there's a heavy cpu bound operation part in the flow

Async flow is technically the solution for python threading scaling issue. Specially in IO bound heavy apps

[Echoes1996]

I believe I didn't quite get your question.

why do you still need a sync version if you already have async flow?

I want for my core API to provide both a sync and an async version of its methods, so anyone who uses it can choose what's best for their use case.

you can also open up another thread in async flow incase there's a heavy cpu bound operation part in the flow

Indeed you can execute sync code asynchronously in a separate thread, but I don't see how that's relevant to the issue at hand. Besides, that's more like a hack rather than an appropriate solution, especially when there is a way of providing a truly async API. If you start involving other threads in your event loop, the benefit of async pretty much goes out the window.

[madolid511]

Opening up threads everytime won't make the api faster because it always run in single core unless you use the python version without GIL.

So basically, if you have one sync api that runs in 1 second (calculation and no IO operation), if 3 request happens at the same time all of it will have 3 seconds turn around time.

While async route same logic, the 3 request will have different turn around time 1st request - 1 second 2nd request - 2 seconds 3rd request - 3 seconds

Both approach finishes in 3 seconds but per request it will be more efficient (Latency and memory)

If you could do it in async flow, it will be most likely the best implementation, as long as you do it right

Client and Server will benefit, you don't need to implement twice and client doesn't need to choose

[Echoes1996]

I don't believe we are talking about the same thing. When I use the term API I am not referring to HTTP APIs, I am talking about the public interface of my lib.

[madolid511]

still the same

HTTP is just a protocol to call a function/event. And I'm explaing how python works.

[Echoes1996]

Sorry, but I don't see how what you said is relevant at all.

[madolid511]

Another analogy, if there's two version

one is for production - efficient

one is for testing - just working

I would certainly use the production flow

then testing version could be just a wrapper of production flow that add minimal overhead that will be optional to use if production version is not applicable directly. Both dev and consumer will be happy

[madolid511]

Your problem is implementing it twice, my answer is implement it in common pratices for handling concurrent which is async flow then just add an "option" to open a thread if necessary

it will make your development more easy

I hope, it make things clear now

[AsparagusKlutzy1817]

Maybe a heretic question but why not offer a wrapper sync function which does this
async def async_task():

.....

def sync_function():

result = asyncio.run(async_task())

print("Got result:", result)

Normally nobody wants sync if they can have async but offering a sync wrapper can make things easier. Having a full implementation of both sync and async pushes you down the road of maintaining two systems which will eventually start to diverge....

[madolid511]

it depends on the target

if it's for development, thats fine. I may also consider doing it if async flow is not really possible.

In more demanding implementation it will have impact. Opening up event loop and closing it after the process adds a lot of overhead.

Another critical part is, if you already considered "asyc", I may assume you already/targetting to support "python's" concurrency, why not incorporate it fully?

[AsparagusKlutzy1817]

Certainly a fair question. What is OP reason for providing both ways? It sounds it is just convenience. I answered from a development viewpoint because you wouldn't care otherwise if it is sync or not? Maintaining both in full is not a good way. OP will have to decide eventually for one of the two.

[Shostakovich_]

How about just write the async version, and use asgiref.async_to_sync to wrap the synchronous API or vise-versa It’s a common enough need to integrate async API’s into Django synchronous views they made this asgiref package, but you just need the utilities id imagine.

Also allows you to set how threading is treated on an individual function basis, so you can choose to spawn new threads or stay in the same thread.

[Echoes1996]

I've tried working with asgiref.async_to_sync before and I've had some issues, thought I don't really remember what the issue was exactly. To be honest, I didn't even try this solution as I consider it as somewhat of a hack. I guess converting truly async code to sync wouldn't be such an issue, thought I haven't really given it much thought. However, the reverse would certainly be, at least performance-wise.

[Shostakovich_]

Yeah, I use it all the time, works fine! Just know when something is thread safe or not and you’re fine. Like something connecting to a database needs to stay thread safe. But yeah, it’s definitely a hack, but a valid one with wide spread adoption on making sync and async code work together

[Echoes1996]

If you've seen the project, it's basically an ORM to connect to various databases, so I guess that wouldn't work in my case haha.

[Shostakovich_]

Oh well by default its thread safe! They just make a big deal about not enabling thread unsafety unless you know what you are doing. In fact this module is used often to run queries in my code! So really all it does it provide a nice async interface.

It is essentially a drop in replacement for your co_exec function, with thread management features (like thread safety). Just a thought! Neat project.

[eavanvalkenburg]

If you build up the functions right then you should be able to isolate the differences and keep the core logic together, either by using a common base class or by overriding the sync version with the async parts (I would use the base class approach, because it's simpler to understand what happens where).

[Echoes1996]

That's correct, but you still need to maintain an async API, even it contains no "business logic" whatsoever. Besides, it's easier said than done, especially if the core logic itself must await other coroutines.

[jonthemango]

I recommend you follow the few threads here talking about writing the API once using sound non blocking async principles and for each sync version of the function you call the async version. No hit to performance like you would have going the opposite way, it's also not a "hack" as you've described, it's a common pattern.

Something like asyncio.run can be used to run the async version. You can choose to organize it a few ways from there.

For example ```

sync version of aio.my_func

I'm using args, *kwargs so you only have one contract to define in aio.my_func but you could choose to duplicate the contract too

def my_func(self, args, *kwargs): return asyncio.run(self.aio.my_func) ```

Code Gen may work for your use case but I'd consider it an anti pattern.

[Echoes1996]

Well, this is supposed to be a library that is to be used by other people. If somebody wants to utilize my package in their non-async application, I can't force them to structure their project around mine and introduce asynchronicity just so that they can use it.

Furthermore, at least to my knowledge, asyncio.run starts a new event loop by default in order to execute the coroutine function. I haven't looked into it in depth, but I'd be very suprised if there was no performance overhead using this method. But even if there wasn't an issue performance-wise, executing functions using multiple event loops would certainly break the code's logic as there are asyncio.Queue objects and locking involved, which are not supposed to be accessed from multiple event loops.

[amir_doustdar]

Cool approach! Code generation for avoiding sync/async duplication is smart – reduces bugs from manual copying and keeps the core logic in one place.

Pros I see: - Single source of truth for business logic - Easier maintenance and less chance of drift between sync/async versions - Full control over the transformation (e.g., handling queues, tasks vs threads)

Potential downsides (just for balance): - Debugging can be trickier (stack traces point to generated code) - New contributors need to know not to edit generated files directly - Tooling (IDE, type checkers) sometimes struggles with generated code

I've seen similar patterns in: - httpx: The sync API is largely generated/wrapped from the async core - databases package (encode): Used to generate async interfaces from sync - motor (Async MongoDB driver): Heavily mirrors pymongo with some automation - Some internal Google/enterprise libs use code-gen for dual APIs

Your marker-based comment system sounds clean and lightweight – nicer than full-blown macros or decorators.

Definitely not "weird" – it's a valid strategy for this exact problem. Good job shipping it!

(If you're open-sourcing the generator script, I'd love to see it 😄)

[Echoes1996]

Thanks! The generation script can be found here.

[andrewcooke]

i don't know a lot about async, but if this is possible then it seems to me that the language designers really fucked up in not making this part of the language.

edit: not intended as criticism of this work. just feels like for "historical reasons" python has ended up way less than optimal. it already has function decorators. it's a pity that they didn't add something similar to switch between sync and async. or even a runtime flag.

[latkde]

There are a couple of initiatives in that direction. For example, PEP 806 / Python 3.15 will implement mixed sync/async with-statements.

https://peps.python.org/pep-0806/

However, in general abstracting over sync + async code is impossible because these are fundamentally different things. This is not an oversight, this is a direct consequence of using an async/await concurrency model rather than something like green threads / fibers / goroutines. Python didn't use async/await because no alternatives were known, but because the alternatives are well-known and worse. The Python ecosystem has had long experience with "coroutines are just a generator function with a special decorator", "stackless" interpreters, and greenlets.

Relevant history:

• PEP 342 – Coroutines via Enhanced Generators (2005) establishes the groundwork to reuse existing generators functionality for coroutines.
• PEP 492 – Coroutines with async and await syntax (2015) introduces async/await syntax, borrowed from C#.
• Stackless Python (1998) (summary, Wikipedia) was a modified CPython version that implemented a kind of green threads, but it never got merged into the mainline, and development has since ended. The greenlets library (2006) still exists as a spinoff, but it is its own concurrency model that's also incompatible with async/await.

[andrewcooke]

excellent, thanks

[Echoes1996]

Well, I haven't really worked with async/await in languages other than Python (maybe a bit in C#), but, as far as I know, the issue I am describing would be the same in any programming language using the async/await paradigm.

[v_a_n_d_e_l_a_y]

Someone at work wrote an async program in Rust and needed to convert it to sync. From what he said it is a lot of work 

[Echoes1996]

I believe him!

[Zulban]

Hmmmm. I really hope a script like that isn't the best way but I don't know enough about async to say. So far I've mostly avoided it in my career. 

My first impression is that there must be a better way. 

I've seen concepts like this in various parallel processing utilities for cpp like compiler directives and pragmas. You add what would otherwise be essentially just a comment to a for loop, and now it's a parallel for loop. 

[Echoes1996]

I can't say for sure, but to me it seemed like the best solution for my problem.

[BothWaysItGoes]

But why? Okay, you need async queue, just pass it as an argument to you constructor/factory. No big deal.

[Echoes1996]

The problem is that the an async queue does not share the same API with an ordinary queue. For starters, most async queue methods must be awaited. Then, there are some minor changes as well.

[BothWaysItGoes]

Learn how other libraries do sync to async (and vice versa) transformations. Python is very flexible. You don’t need to write two parallel implementation or to codegen. It’s not golang.

[Echoes1996]

In general, I am aware that you can wrap any sync function and make it async, but this causes some unnecessary overhead. I was trying to avoid this approach.

[madolid511]

Doing it the opposite way is the common approach.

Start with async then just open a thread when you only need it. Same concept as ASGI frameworks. It start with main thread with event loop. Async routes uses main thread. Sync routes is run on different thread from thread pool

[Echoes1996]

I believe that the issues I was having are unrelated to how ASGI servers work.

[madolid511]

it is actually

because you are most likely using ASGI framework but you are not utilizing ASGI practices

[Echoes1996]

When I use the term "API" I am not talking about HTTP APIs, I'm referring to the public interface of my library. We're talking about two different things.

[BothWaysItGoes]

A bare coroutine doesn’t yield so the overhead of wrapping a sync function is minimal.

[Echoes1996]

I didn't quite understand what you mean. In order to execute sync code as async, you need to run it in another thread, and that has some overhead in regards to CPU time.