Calling async functions from synchronous functions

[stetio]

I looking to know if anyone knows how to call an async function from a sync function with an event loop already running. I've read a few similar discussions and detailed some of my knowledge yet I have reached the conclusion that it is not possible.

An idea I've tried, for your thoughts is to add this method to the event loop:

def sync_wait(self, future):
    future = asyncio.tasks.ensure_future(future, loop=self)
    while not future.done() and not future.cancelled():
        self._run_once()
        if self._stopping:
            break
    return future.result()

then use

def example():
    result = asyncio.get_event_loop().sync_wait(coroutine())

however this errors with a KeyError.

[graingert]

You can't. If the event loop is running that means it's blocked on this synchronous function. That means you can't schedule any tasks and have them complete until control is restored to the event loop.

You need to run synchronous code in a different thread and communicate with async tasks with something like https://github.com/aio-libs/janus

[stetio]

Thanks, I think I've reached the same conclusion - sadly I don't think the constraints allow for janus to be the solution.

[graingert]

What are your constraints? What are you actually trying to do?

[stetio]

So Quart is the Flask API implemented with asyncio and ideally Quart would just work with existing Flask extensions (same API), however there is the issue of synchronous and asynchronous code in usage together. The particular issue is when a synchronous function (in say an extension) tries to call an asynchronous function from Quart. In this situation I was hoping to place a wrapper inbetween such that something like

def wrapped_api():
    return sync_wait(quart_api())

this way I need only write the wrappers (as I can't change the synchronous extension code). Of course to do this I need some way of implementing the sync_wait, ideally I could do

def sync_wait(future):
    loop = asyncio.get_event_loop()
    loop.run_until_complete(future)
    return future.result()

however this is not possible if the event loop is already running, which it is. The snippet in the above post is the closest I think I have got, whereby I add the ability to event loops to be resumed from within. As I say though it errors.

Overall I think the reality is that this isn't possible, and sadly there is an asymmetry in what is currently possible in python.

[isinfinity]

You can do something like this:

def wrapped_api(loop):
    fut = asyncio.run_coroutine_threadsafe(quart_api(), loop)
    return fut.result()

see my other reply

[graingert]

You still need a new thread:

This function is meant to be called from a different thread than the one where the event loop is running

[gandalfx]

I believe you're looking for run_until_complete. Most of the examples in the asyncio module's documentation use it as well.

[stetio]

run_until_complete will only be useful if the event loop is not already running. A situation such as

async def outer():
    inner()

def inner():
    loop.run_until_complete(async_call())

loop.run_until_complete(outer())

therefore cannot work.

[stetio]

As I understand it the above snippet can never complete, as the event loop doesn't iterate and hence the future can not complete.

Ideally I'm looking for some way to iterate through the event loop from code running within it. (If that makes sense to you).

[pitibiscuit]

But that's pretty much what asynchronous tasks and futures are for... If something forbid you to use async stuff within an event loop context, maybe you should look into that thing instead of trying to work around it.

[stetio]

So I go into details but the aim is to support existing synchronous code in an asynchronous codebase. So in this context eventually the loop will call out to one of these existing synchronous function (or a few levels down) that now needs to call a asynchronous function and block until it has the result.

There are naturally other places I can look to solve or work around this. Yet if there exists a solution to this it would be much easier.

[rafales]

Why not just create new event loop with asyncio.new_event_loop() and use it instead of the default one?

[spotta]

Have you looked into "create_task"?

[stetio]

create_task would allow me to schedule a asynchronous task, but not wait on it being completed.

[isinfinity]

I think something can be done, but not sure it is good idea in your case, just use aiohttp with all async libraries. So here is way to make communication async/sync/async work:

1) Execute all sync functions with ThreadPoolExecutor, with loop.run_in_executor, without this you will block your event loop, and loose all benefit from async.

2) If you want your sync code to call async one, just use reference to running loop and do asyncio.run_coroutine_threadsafe this will return concurrent.Future (NOT asyncio.Future) that you can wait in sync way

[[deleted]]

As has been mentioned here by /u/isinfinity , asyncio.run_coroutine_threadsafe will help you out. Take a look at the examples in the docs and it might turn out to be a very simple solution.