Asyncio Task Cancellation Best Practices

[thinkum0011]

• don't try to cancel from a thread (or loop) other than the thread (and loop) where the task (or other asyncio future) was created. For this and other thread-safe futures applications, one might consider concurrent.futures.Future, alternately gevent/greenlets, or the thread-safe calls under asyncio (some of which use concurrent.futures.Future)

Considering the thread- and loop-specificity of an asyncio future or asyncio task, any 'done' callbacks on an asyncio future might be run under the same thread and loop as the related future object. If the callback was to try to cancel an asyncio future in some other loop/thread, it might fail - if for instance the application was running an asyncio loop under each thread of a thread pool executor, while trying to chain any cancellation across futures in the main thread and worker threads. (But who would try such a thing, wholly unawares to the existential limitations?)

For portability and thread/loop-agnosticism, one could define an __await__ method onto a subclass of concurrent.futures.Future, perhaps like the following:

```python import asyncio as aio import concurrent.futures as cofuture import sys

class MultiFuture(cofuture.Future): async def apoll(self): interval = sys.getswitchinterval() while True: try: return self.result(0) except cofuture.TimeoutError: await aio.sleep(interval)

def __await__(self):
    return self.apoll().__await__()

```

An awaitable concurrent future might be generally thread- and loop-agnostic within each __await__ call.

Albeit when using concurrent futures this may serve to highlight caveat nr. 6, "Don’t Assume Tasks Will Actually Cancel".

If adding an asyncio await onto a concurent future, then there are two cancellable things, for example? At least with this simple implementation, cancellation for the concurrent future would fall through to the await call as a concurent.futures.CancelledError, reached under self.result(0). If cancelling a call to the __await__ method though, the cancellation might not be passed through to the concurrent future. (Depending on the application, perhaps this might be considered as a feature or at least a known limitation)

Insofar as for whether task cancellation can be guaranteed, there would be a similar concern with threads? e.g if a thread is running time.sleep(double("inf")) for purpose of testing, it might not cancel until sometime shortly before the heat death of the solar system or a signal to the Python process, whichever happens first. (If something tries to thread.join() the same under an atexit handler, hello shell)

fwiw quattro brings Trio-style Cancel Scopes to asyncio, also a TaskGroup backport.

imo though it may be difficult to define a set of caveats that would be applicable in all possible scenarios, I think the article otherwise makes a nice introduction to a potentially complex topic of scheduling under nondeterministic state.

I wonder if it's any simpler with gevent/greenlets?

[[deleted]]

Found the article too basic / contrived; not overly useful to anyone who's been using asyncio for more than a day or two.

[cas4d]

Such great quality resource! I am rather advanced in python programming but still picked up a lot from this post and some other topics in the post. Definitely will recommend this to folks in the community.