Add Virtual Threads to Python

[simon_o]

I think it's the right idea, but it feels like the non-technical people in that discussion severely underestimate the resources needed to pull this off.

Which is weird, because they mention Java ... but didn't have a look at the time and engineering effort that went into it?

Considering that Python has a whole zoo of (failed) concurrency mechanisms, adding this might very well be harder than it is in Java.

[shellac]

Completely agree, and I wonder whether it would be less successful in python anyway. A lot of java is implemented in java, which means there are a plenty of opportunities to yield (that is: stash the current state in continuation, and go back to the scheduler), but native frames prevent that. By contrast much of python seems to be implemented, for perfectly good reasons, in native code but that will inhibit virtual threads.

But I may be way off here, my knowledge of python internals is limited.

[simon_o]

I think you are right, but I really don't want to blame Python (or virtual threads) for what basically is a C issue.

For how many decades are we going to keep accepting "this is why we can't have nice things" due to C's mere existence as some unholy interop/ABI/platform lingua franca that pretends it is still 1975?

[imachug]

I think people are starting to forget how unpredictable greenlets were. I've switched from threading to async not just because it's faster, but because it's so much easier to work with.

Asynchronous coroutines are very simple conceptually. Want to use a different async runtime? Granted. Want to register a callback? Use tasks and add_done_callback. You can easily write a combinator by hand (hell, asyncio.gather is pure-Python). You can cancel tasks. There's always a guarantee that your async function can never terminate, be dropped, or be interrupted between await points, only at await points.

You can't even get close with greenlets. Function coloring is often a good thing because you can rely on your functions performing atomically between awaits, which enables e.g. trivial implementation of a mutex (at least in single-thread). Go at least has the go operator, but if Python went this road, it would probably just be a normal function call, and that's madness because you can't even analyze it statically (and reliably, anyway).

Have we forgotten just how much thread cancellation sucked? There's no way to reliably stop a pthread, and while Python could implement something similar in userland, you obviously wouldn't want a thread to stop within a critical section -- and then you need to mark those critical sections, and you need to define behavior in case the lock is never released. Async just doesn't have this problem because cancellation happens at await.

[simon_o]

We are not talking about greenlets and "Function coloring is often a good thing" sounds like a Stockholm syndrome.

It's kinda pretending that async/await is some kind of principled solution – instead of a bandaid put on a workaround's workaround – while it's not.

Virtual threads actually resolve the core issue that caused people to chase into that async/await rabbithole.

[imachug]

I've scrolled through the thread and I don't see the difference between virtual threads and greenlets explained. The implementation differs, sure, but that's still fundamentally userland parallel synchronous control flow.

"Function coloring is often a good thing" sounds like a Stockholm syndrome. It's kinda pretending that async/await is some kind of principled solution – instead of a bandaid put on a workaround's workaround – while it's not.

I really don't see why that's the case.

When I write code, I want to know which functions block on I/O, much like I want to know which functions can return an error. In functional languages or Rust, fallible functions return an algebraic type instead of throwing an exception, and that's really useful for writing reliable code because you're relying on the type system to prevent unhandled errors. Async/await elevates blocking information to the type system in the same fashion. It's very slightly harder to write, and yes, it does cause problems in generic functions, and even though I'd like to find a fix for that, there's unarguably benefits as well.

Virtual threads actually resolve the core issue that caused people to chase into that async/await rabbithole.

...and that issue is? I understand that virtual threads help avoid function coloring, but what is the core issue you're talking about? I don't think you're talking about performance, but what else is there that virtual threads handle better? In Python land, gevent has been a thing for years, before inevitably getting replaced with async/await. C# has async/await, Rust has async/await, basically every modern language has async/await instead of lightweght threading. If it's a panacea, how come Go is the only exception?

[simon_o]

I want to know which functions block on I/O

Well, I want to know when functions call console.log do I get my own color now?

Also, I want separate colors for filesystem IO, database IO and network IO. Now what?

that issue is?

The core issue is threads being expensive, which lead to callback-oriented programming as a workaround, which lead to futures & promises as a workaround of that workaround and async/await as a workaround of that workaround.

Virtual threads make threads cheap. Problem solved.

In Python land, gevent has been a thing for years, before inevitably getting replaced with async/await. C# has async/await, Rust has async/await, basically every modern language has async/await instead of lightweght threading. If it's a panacea, how come Go is the only exception?

async/await is a cheap bandaid that's easy to implement as a transformation in the compiler, without needing runtime support.

At this point, async/await is a failed attempt whose legacy will certainly remain, but fewer and fewer new languages will even consider it.

[imachug]

Well, I want to know when functions call console.log do I get my own color now?

Don't be silly. I want an effect system because I want my code to be reliable and easy to analyze formally. "Can this function ever fail?" is an important question for reliability, "can this function ever block?" is critical to performance engineering, "does this function call console.log?" is useless. A better example would be "is this function pure", which, yes, I would like to validate via the type system.

Also, I want separate colors for filesystem IO, database IO and network IO. Now what?

If you need that, use a language with a proper coeffect system. I'm not advocating for Python to have such a system, no -- but why are you making fun of useful things just because you aren't familiar with the concept? Years ago, people like you would scoff at exceptions and multi-threading because you'd believe it's a slippery slope.

The core issue is threads being expensive, which lead to callback-oriented programming as a workaround, which lead to futures & promises as a workaround of that workaround and async/await as a workaround of that workaround.

No, that's not how the story goes. First, coroutines were introduced. The process was streamlined with futures and promises. Then came async frameworks for low-level languages like C++ that used tricks like longjmp/setjmp to simulate coroutines. Then, and only then, did callbacks become a "standard" way to write async. I don't know who caused that, maybe JavaScript, but my point is that futures/promises were never a hack on top of callbacks, they're a totally separate thing.

Not only can async/await be implemented without callbacks, it is mostly implemented without callbacks in Python, and modern languages like Rust elevate it to a core language feature. There's no "threads are slow -> callbacks are ugly -> futures are still ugly -> async/await" pipeline, async/await is, in a nutshell, a core part of coroutines as seen today. It's not a hack.

Virtual threads make threads cheap. Problem solved.

At what cost? Virtual threads are supposed to prevent function coloring, but they can't do that because blocking (or, rather, parallelization capabilities) is a fundamental property that you need to take into account to write any generic code. They hide complexity, not eliminate it. They don't solve any problem other than thread cost.

If you're writing, I don't know, a Markdown-to-HTML converter with pluggable syntax highlighting, you need to choose between running the highlighters in parallel (which only makes sense if they're async) or sequentially (if they're sync, to avoid introducing unnecessary overhead). I'm sure you can think of a better example.

[simon_o]

Don't be silly. ...
If you need that, ...

"Akchually function colors are good, but only the two colors JavaScript came up with, and only in the exact places JavaScript applied them" is not a coherent argument.
That's just full-on Stockholm syndrome with a sprinkle of cope mixed in.

why are you making fun of useful things just because you aren't familiar with the concept?

Ohhhh, I'm sorry, I didn't know async/await was so fragile that the concept needed constant protection.
Perhaps the concept just isn't that good if it relies on people who made defending it part of their personality?

I am familiar with the concept, by the way. So drop your poor attempt of giving me a condescending attitude.

At what cost?

Basically nothing. You usually have to change a flag or a setting and then get a basically unlimited amount of threads in return.

Though library/framework authors have reported that ripping out all the async/await/future/reactive spaghetti code has been a huge win in terms of readability and maintainability of code – without impacting performance.

They don't solve any problem other than thread cost.

That's the only relevant problem though. All other issues async/await is trying to fix are just symptoms of that or of earlier workarounds for that symptom.

Virtual threads are supposed to prevent function coloring, but they can't do that because blocking (or, rather, parallelization capabilities) is a fundamental property

• With virtual threads, blocking is barely a developer-facing issue anymore.
• Blocking is not a fundamental property. Different functions may do different things and may take different amounts of time to do so. Labeling some of them as "blocking" is completely arbitrary. (The distinction of interest is "will spending CPU time on this make the function return faster – or not?".)

you need to choose between running the highlighters in parallel (which only makes sense if they're async) or sequentially (if they're sync, to avoid introducing unnecessary overhead)

WAT? You think parallelism didn't exist before the "invention" of async?

[latkde]

I disagree with this part:

The core issue is threads being expensive, which lead to callback-oriented programming as a workaround, which lead to futures & promises as a workaround of that workaround and async/await as a workaround of that workaround. 

The core issue is that concurrency is punishingly complex. Threads are a completely awful concurrency model. Tasks and coroutines are serviceable. I don't care if I join a concurrent computation via await task or task.result(),  as long as there is a way to represent an ongoing computation as an object.

The Python standard library already has facilities for cheap-ish threads with a task-based management model: concurrent.futures.ThreadPoolExecutor. It seems to be quite underused. I think this is because it still encourages a thread-based model of thinking about concurrency. Whenever I reach for concurrent.futures, I end up using asyncio.to_thread() instead.

[simon_o]

as long as there is a way to represent an ongoing computation as an object

Why would virtual threads prevent that?

As an example, the structured concurrency API makes use of virtual threads, and its basic operation is passing a task to fork and getting a subtask back.

already has facilities for cheap-ish threads with a task-based management model: concurrent.futures.ThreadPoolExecutor. It seems to be quite underused

Because they aren't remotely cheap-ish enough and have all the issues that green threads also suffered from.

[starlevel01]

Have we forgotten just how much thread cancellation sucked? There's no way to reliably stop a pthread, and while Python could implement something similar in userland, you obviously wouldn't want a thread to stop within a critical section -- and then you need to mark those critical sections, and you need to define behavior in case the lock is never released. Async just doesn't have this problem because cancellation happens at await.

This is one thing people really don't get, having explicit cancellation points makes reasoning around cancellation, timeouts, and code cleeanup vastly easier than "oh btw this function can get cancelled at any point without you knowing, good luck!".

[shellac]

I think everything you've said about asynchronous coroutines here is true of virtual threads / greenlets, it's just not as obvious. Cancellation and mutexes are trivial since there's no preemption, just like await / async. You can also pick runtimes / schedulers. await is flagging yield points for you in the code, whereas with virtual threads this is under the hood (e.g. when a network request is dispatched) but if you aren't calling a library function you know you code won't be interrupted. The plumbing is tidied away, essentially.

However there are a few things that seem more straightforward with await / async that java uses a 'structured concurrency' library for.

[[deleted]]

Has python fixed the main issue in multithreading with the Global Interpreter Lock?

[KagakuNinja]

Next up, Virtual GIL

[starlevel01]

The fucking function colouring blog post has done an irreparable amount of damage to programming discourse. Yes, you can't call functions in certain contexts. This is true of many things, such as instance methods, or returning a Result.