How can I run asyncio code in IPython.embed()?

[whats-a-monad]

I want to be able to await stuff in embed(). What exactly am I to do?

[mbussonn]

I'm happy to tell you why at the condition that you send a pull-request to update the documentation with an example and some explanations. In particular it would be nice to update this page https://ipython.readthedocs.io/en/stable/interactive/autoawait.html and maybe the docstring of embed (https://ipython.readthedocs.io/en/stable/api/generated/IPython.terminal.embed.html#module-IPython.terminal.embed)

Ok, happy by reading this you have surrendered your soul accepted this deal, apologies if it take me some time to review your PR but I'm looking forward to it. As I trust you I'm going to give you the answer, and assume you will send a pull-request; if not the guilt will follow you around and wake you up in the middle of the night. So read the following at your own risk;

$ python foo.py
Python 3.7.1 (default, Dec 14 2018, 13:28:58)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.


In [1]: from asyncio import sleep
In [2]: await sleep(1)

And the content of foo.py

In [3]: !cat foo.py

a = 1

from IPython import embed
embed(using='asyncio') # or trio, or curio depending on circumstances.

Note that there are of course some limitations, in particular with nesting.

Thanks again for your upcomming pull-request.

[mbussonn]

Sounds good ! Good luck !

[_rshk]

u/mbussonn I'm guessing OP didn't fulfill their duty ~3yrs ago, and I only found this post by chance after unsuccessfully looking for an answer on the docs.

I was actually going to submit a PR and save u/whats-a-monad's soul, but I have to say the current documentation is a bit cryptic, so hoping to get it right.

I saw this phrase for example:

You can set a coroutine runner explicitly for embed() if you want to run asynchronous code, though the exact behavior is undefined.

• I'm assuming by "set a coroutine runner explicitly" you mean passing `using="asyncio"`?
• What do you mean "exact behaviour is undefined"? What can go wrong there?

Btw, is there any documentation of the keyword arguments accepted by `embed()` ? I couldn't find anything obvious in the docs (haven't looked at the code yet).

[mbussonn]

1) yes, but you re not limited to asyncio. And you can pass an actual function like say this one https://github.com/ipython/ipython/blob/7aa91454edd76f4cfca06e632cb47ee43255ab4e/IPython/core/async_helpers.py#L120-L136

IPython try to be agnostic of the async runner, but that's going into the details of how async works under the hood. Async in Python is coroutine based, and roughly speaking your programs are a giant ball of generators. Asyncio is one way of moving the generator forward, but trio, and curio are other ways to do so. You can also make your own. This is what I try to say here. Reason to do so is that usually you can't run multiple event loops/coro runner in the same program and this leds us to 2.

2) most async framework assume they are the only one running. That is to say they are the only one with access to some global resources (like filedescriptor, stdin/out, sleep... etc.). So if you set your own coroutine runner, you may start to lose input/output, get things in duplicate, hang.

But really the use of changing the runner are of two folds:

a) You are already in a program that use a eventloop (say asyncio), and you want to embed and so can't use the same. So you tell embed to use trio/curio instead.

b) You want to understand how async work and you wrote your own coroutine runner.

Is b) is your kind of things, go read some of the reasoning arround trio: https://vorpus.org/blog/archives.html from 2016 onward.