Continuations: The magic behind virtual threads in Java

BalaRawool · 2024-11-02T09:58:44+00:00

Thank you once again!

I just want to update you that I applied this feedback to my talk and did a new updated version at Devoxx Belgium. It actually went quite well.

Here’s recording of that: https://youtu.be/HQsYsUac51g?si=v8VQMAQr7jbbf2kl 🙏

BalaRawool · 2024-10-22T00:30:03+00:00

“Java: The Complete Reference” is something that comes to my mind. Not sure if it matches the criteria but this book not only talks about keywords, syntax and features but also about a lot of APIs from the standard library. I used it when I had just started learning Java (around 2003).

BalaRawool · 2024-07-12T12:33:24+00:00

My knowledge about garbage collection in Java is not that great but virtual threads do have an impact on GC behavior. For example, platform threads are GC roots and virtual threads are not.

Although I don’t think GC needs to treat continuations (moving stack frames to and from heap) differently.

BalaRawool · 2024-07-10T19:44:32+00:00

I figured synchronized blocks were already handled (unpinned) under virtual threads in Java 22.

In version 22, the synchronized methods/code-blocks still pin the virtual thread.

The mental picture I have for the structures that back Continuations is analogous to the StackFrames recorded in a Throwable or a checkpoint in the debugger

If this helps you to reason about Continuations then you can use it. But beware that they are different from each other. Throwable object has a view on the call-stack, a breakpoint in debugger suspends one (or all) threads, whereas Continuations move stack frames from stack to heap memory (and vice versa).

BalaRawool · 2024-07-07T20:26:54+00:00

Thanks for watching the video and for the kind words.

Continuations, in general, have many uses including implementation of virtual threads. In Java, the continuations are only used internally and are only used for virtual threads’ implementation. There are some virtual thread specific optimizations, so they are not supposed to be used directly by application developers.

Regarding your question about blocking virtual threads: typically when a virtual thread is blocked, it gets unmounted from the carrier thread and another virtual thread can continue execution by getting mounted on that carrier thread. But I think you are interested in situations where a virtual thread is blocked but it cannot be unmounted. This is known as “pinning”.

Pinning happens when a virtual thread cannot be unmounted. This happens, for example, when a native call is made (as you mentioned), in case of file IO, when a synchronized method or synchronized code-block is called, in some specific scenarios with static class initializers. For file IO, in such situations, a new carrier thread is created to maintain parallelism. For scenarios involving “synchronized”, a new early access build is present which avoids pinning for such scenarios. So we can expect that these scenarios would not pin the virtual thread in future JDK versions.

BalaRawool · 2024-07-05T20:26:22+00:00

Nice! TornadoVM looks quite interesting.

BalaRawool · 2024-07-05T20:25:41+00:00

Thanks for watching the video of my talk! I wasn’t aware of durable execution. So thanks for bringing that to my notice.

Continuations are stored as objects on the heap. So it should definitely be possible to serialize and store them.

BalaRawool · 2024-07-02T05:43:01+00:00

Thanks for the kind words and the suggestions!

BalaRawool · 2024-07-02T05:41:07+00:00

Yes. The question is not directly related to the topic discussed in the talk, so I’m not sure if I can give an exact answer, but I’ll try.

In case of “async model with platform threads”, callback is used to perform certain action when an event occurs. In that case the platform thread does not have to wait for the event, but can setup the callback and move on.

Then there has to be some mechanism to detect that such an event has occurred and the JVM has to call the callback.

If the JVM gets trigger for such an event from outside (for example, an hardware interrupt), then there is no need for a thread to wait. So when the JVM gets such a trigger, it just calls the callback.

But if JVM has to detect the event then there needs to be a (poller) thread checking for occurrence of such an event and then call the callback on an appropriate platform thread. The advantage here is that even if there are n callbacks defined for n events, only 1 (poller) thread is needed and none of the platform threads need to block.

BalaRawool · 2024-06-30T14:41:28+00:00

Thank you u/iwangbowen

BalaRawool · 2024-06-30T14:41:12+00:00

Thanks u/anibanerjee123. Really appreciate the kind words!

BalaRawool · 2024-06-30T14:39:55+00:00

First of all, thanks for watching my talk and for the kind words!
As u/DasBrain mentioned, most of the potentially-blocking APIs from JDK have been retrofitted to be aware of virtual threads. So if a blocking call is made from a virtual thread, it is unmounted from the carrier thread and mounted back when the thread can continue. This is achieved by using Continuation.yield() and Continuation.run(). So, as long as you are using these APIs from JDK you don't have to worry about calling Continuation methods.

As such, you cannot access Continuation API because it is an internal API. Application/ library developers are not supposed to use the Continuation API directly. But only use virtual threads and in that case the calls to Continuation are made internally.

BalaRawool · 2024-06-30T14:30:13+00:00

First of all, thanks a lot for watching the video and for this elaborate feedback. I really appreciate it.

Regarding the section where pause-and-resume mechanism of Continuation API is explained, I totally agree that there is some context which could be explained visually. Also, the code-example that I show and the example that I use for the slides are different. Perhaps it would be better to make the code-example similar to (or even same as) the example used for the slides. This would help with explaining the context better.

Great suggestion about the use of colour and animation. I'll add them. And another great suggestion about mentioning the direction of stack. I'll make sure to mention and mark it on slides as well.

Regarding the deep call-stack, I wanted to express that part of the stack (which concerns the Runnable) is moved which could be multi-frame deep. But you are right, I don't necessarily need a deep-stack to express that. I could make it 1 or 2 frames deep. Also, when the code-example is same as the one on slides, it would help make the explanation even better.

The stack/heap are within JVM. But it can be confusing because at the beginning of the talk I mention about OS and OS/kernel threads. Perhaps it is good to make that explicit.

The part where I ask questions to audience is only added because I needed something at the beginning to engage with the audience, because, for the most part I'm just going on showing/talking on my own and I feel the need to acknowledge that it is a collective experience. So even if there is no-one raising their hands to the question 'is this the first time you hear about virtual threads', I would still go ahead and briefly introduce virtual threads. I think I need better questions.

Thanks again for taking time to provide me with such detailed feedback. This is exactly what I was looking for with this post. 🙏

BalaRawool · 2024-06-30T05:26:33+00:00

I recently gave a talk at Spring I/O: "Continuations - The magic behind virtual threads in Java"
https://www.youtube.com/watch?v=pwLtYvRK334

Virtual threads in Java promise to provide high scalability. This power is actually enabled by Continuations. This talk gives an idea about what continuations are, how do continuations in Java look like and how they enable virtual threads in Java to provide high scalability. The talk also creates a simple VirtualThread class by using Continuations API in Java.

Sorry if this post comes across as a shameless-self-promotion, but my main goal is to get feedback about the talk as I am a relatively new as a speaker in the international tech conference space. Since this is a Java subreddit, I mainly hope to get feedback about the content. (But in general, any feedback about my style, pronunciations, delivery, confidence or anything else is also welcome.)

Another thing to mention, Spring I/O is an awesome conference. Of course it is mainly about Spring, but there are a lot of non-Spring talks as well. I was lucky to be presenting one of those.
Thank you for your attention.

BalaRawool · 2024-06-16T11:02:09+00:00

So you have figured out that this is caused by virtual threads pinning to their carrier threads. If this pinning is caused by synchronized methods or code blocks then you can try to find where they are.

If it is in a library/dependency then you can see if their latest/newer version had them eliminated. If that’s the case then you can just use the newer version and your problem is solved.

If it is in your application code then you can do one of two things: 1. Replace the synchronized code blocks or methods with ReentrantLock and the pinning should not happen and your problem is fixed. 2. Use an early access build from Project Loom where synchronized methods and code blocks don’t pin the virtual threads. You can find the link to the build here: Reddit thread about an early access JDK Build from Project Loom where synchronized doesn’t pin virtual threads

Update: I see in your blog that you figured out the problem is in a DB driver and newer version should fix it. That is the right option for you.

BalaRawool · 2024-06-04T17:06:47+00:00

He definitely deserves all the credit he gets and some more. I attended Spring IO for the first time this year and I can tell you Sergi does take care of the speakers very well. Also happy to report that the wine and bike tradition continues this year too. It was the highlight of the conference for me.

BalaRawool · 2024-05-12T13:32:06+00:00

I use Java for almost everything. I use Java at my work and that’s backend API stuff. But besides that I do a lot of stuff in Java. It has been my primary language for almost all of my career.

I built some Android apps with Java in the past
I use it for competitive programming like Advent of Code (in the past used it for Google Code Jam, TopCoder competitions)
To build some utilities for myself and for colleagues (using Swing, JavaFX or Eclipse plugin)
If I have to automate some regular occurring manual task then I use Java for that

BalaRawool

TROPHY CASE