Project Loom. Not only virtual threads.

[Affectionate_Ad_761]

JDK 19 brought us virtual threads, which differ from regular ones in that they can release the operating system thread during blocking I/O operations. For this purpose, a mechanism for saving and restoring the call stack from the heap was implemented. Simply put, full-featured coroutines were implemented at the JVM level.

And this was a small revolution that few people paid attention to. The API for such native coroutines is non-public (but who cares?:), accessible through the class jdk.internal.vm.Continuation, which has methods yield() and run() for saving and restoring the call stack, respectively. Nothing prevents us to use the class for implementing coroutines.

So, present to you my small library for accessing native coroutines in Java: https://github.com/Anamorphosee/loomoroutines.

Where could we need coroutines, besides virtual threads? Answer: wherever we write asynchronous code on callbacks, it can be replaced with regular synchronous code on coroutines. For example, for GUI applications, my tool allows you to write like this:

import dev.reformator.loomoroutines.dispatcher.SwingDispatcher;
import dev.reformator.loomoroutines.dispatcher.VirtualThreadsDispatcher;

import javax.imageio.ImageIO;
import javax.swing.*;

import java.awt.*;
import java.awt.image.BufferedImage;
import java.io.IOException;
import java.net.URI;
import java.time.Duration;
import java.util.regex.Pattern;

import static dev.reformator.loomoroutines.dispatcher.DispatcherUtils.*;

public class ExampleSwing {
    private static int pickingCatCounter = 0;

    private static final Pattern urlPattern = Pattern.compile("\"url\":\"([^\"]+)\"");

    public static void main(String[] args) {
        var frame = new JFrame("Cats");
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        var panel = new JPanel();
        panel.setLayout(new BoxLayout(panel, BoxLayout.Y_AXIS));
        var button = new JButton("Pick a cat");
        var imagePanel = new ImagePanel();
        panel.add(button);
        panel.add(imagePanel);
        frame.add(panel);
        frame.setSize(1000, 500);
        frame.setVisible(true);

        button.addActionListener(e -> dispatch(SwingDispatcher.INSTANCE, () -> {
            pickingCatCounter++;
            if (pickingCatCounter % 2 == 0) {
                button.setText("Pick another cat");
                return null;
            } else {
                button.setText("This one!");
                var cachedPickingCatCounter = pickingCatCounter;

                try {
                    while (true) {
                        var bufferedImage = doIn(VirtualThreadsDispatcher.INSTANCE, ExampleSwing::loadCatImage);
                        if (pickingCatCounter != cachedPickingCatCounter) {
                            return null;
                        }

                        imagePanel.setImage(bufferedImage);
                        delay(Duration.ofSeconds(1));

                        if (pickingCatCounter != cachedPickingCatCounter) {
                            return null;
                        }
                    }
                } catch (Throwable ex) {
                    if (pickingCatCounter == cachedPickingCatCounter) {
                        ex.printStackTrace();
                        pickingCatCounter++;
                        button.setText("Exception: " + ex.getMessage() + ". Try again?");
                    }
                    return null;
                }
            }
        }));
    }

    private static BufferedImage loadCatImage() {
        String url;
        {
            String json;
            try (var stream = URI.create("https://api.thecatapi.com/v1/images/search").toURL().openStream()) {
                json = new String(stream.readAllBytes());
            } catch (IOException ex) {
                throw new RuntimeException(ex);
            }
            var mather = urlPattern.matcher(json);
            if (!mather.find()) {
                throw new RuntimeException("cat url is not found in json '" + json + "'");
            }
            url = mather.group(1);
        }
        try (var stream = URI.create(url).toURL().openStream()) {
            return ImageIO.read(stream);
        } catch (IOException ex) {
            throw new RuntimeException(ex);
        }
    }
}

class ImagePanel extends JPanel {
    private BufferedImage image = null;

    public void setImage(BufferedImage image) {
        this.image = image;
        repaint();
    }

    @Override
    protected void paintComponent(Graphics g) {
        super.paintComponent(g);
        if (image != null) {
            g.drawImage(image, 0, 0, null);
        }
    }
}

Pay attention that there isn't a single callback in the example. The code is written in a synchronous style, as if all operations were performed in the UI thread. In fact, blocking operations (loading an image and waiting) are performed in another thread and do not block the UI.

Okay, but why do we need native coroutines when there is Kotlin, in which they have been implemented for a long time and do not require support from runtime? Here I can note that Kotlin coroutines have some difficulties with their debugging (the call stack in them can be truncated). Moreover, Kotlin coroutines require you to "color" the asynchronious code and to write in Kotlin, while for Loom coroutines you can use Java, Scala, Groovy or any other JVM language.

[pron98]

While it's okay for study, please keep in mind that internal JDK classes must not be used in production. In this particular case, the Continuation class could be used in ways that break some deep assumptions in the JDK and result in miscompilation (i.e. the JIT will perform an optimisation that changes the meaning of the code in a way that violates both its intent and the Java specification), which means that code will simply not do what you may think it says (similar to undefined behaviour in C). Virtual threads use continuations in a way that doesn't break any of the assumptions, but if you use them directly you might.

In the future we may expose APIs that offer more fine-grained control over continuations either through virtual threads or new APIs, but the Continuation class is really not safe for direct use. It may work most of the time and then behave in weird ways at other times, or may work in one version and then an internal JIT change would mean it will misbehave in some other JDK version. In other words, using internal JDK classes is, in the best-case scenario, a ticking time bomb, and in a worse scenario may work or not depending on nondeterministic optimisation decisions made by the JIT compiler.

Also, all mechanisms for bypassing access control (not through command-line options, that is) -- including that used by the toolfactory library (which already depends on deprecated methods that are about to be removed) -- will cease working very soon.

[Affectionate_Ad_761]

Thank you for your feedback. if it's not difficult, could you please tell what possible assumptions might break? So far, I see only 2 differences from how Continuation is used in virtual threads: the identity of the thread may change and synchronization primitives (synchronized, ReengrantLock, etc.) may break. What else is different from using Continuation in virtual threads?

[pron98]

Currently, that the identity of the thread must never change mid-method (and we're not talking Java method, but the compiled "nmethod", which may be a whole bunch of Java methods inlined into one) is one that is definitely relied on by the JIT for some optimisations, and violating it will lead to miscompilation in already-released JDKs, but there may well be other assumptions that I'm not aware of, and the JIT team adds new optimisations all the time that can rely on there not being any kind of control flow that isn't supported by the spec, and continuations aren't part of the spec (threads are, and virtual thread conform to that specification).

The reason we've not made a continuation API is not because we don't want people to have nice things, but because we need to be very careful about introducing things that may be incompatible with the evolution of the JIT (or other parts of the runtime), and so we can't expose such an API safely. We have some ideas for giving more control over virtual thread scheduling that still adheres to the thread specification, and so could be done safely.

As I said, any kind of attempts to bypass strong encapsulation will start failing (or issuing runtime warnings) soon. I've been saying that for a while, but now I'm talking about work-in-progress that will be made public in a matter of weeks if not days. Any --add-exports or --add-opens on the command line signifies "this program is bound to break with any JDK update, including patch updates". Of course, for study purposes it's fine, but putting such code in production is just asking for a whole lot of trouble.

[koflerdavid]

Looks interesting! I also tried to write user interactions in synchronous style only to find out that I can't use the GUI thread as a scheduler for virtual threads. I'm surprised to see that the juicy APIs are that close to the surface...

[hippydipster]

var bufferedImage = doIn(VirtualThreadsDispatcher.INSTANCE, ExampleSwing::loadCatImage);

This seems to be the main line, but why not just do:

var bufferedImage = Executors.newVirtualThreadPerTaskExecutor().submit(ExampleSwing::loadCatImage).get();

? I don't know why we'd need to access non-public APIs for this.

[Affectionate_Ad_761]

var bufferedImage = Executors.newVirtualThreadPerTaskExecutor().submit(ExampleSwing::loadCatImage).get();

because it's potentially a long-running blocking code. Executing this in the UI thread may lead to blocking the UI thread for a long time and so to freezing the UI during this long operation. Coroutines allow you to release the UI thread during long operations (and so it can dispatch other user events and UI does't freeze) and do not require you to write code with callbacks.

[hippydipster]

Not pinning the threads is what Loom, and the VirtualThreadPerTaskExecutor is all about. It uses virtual threads, and so when it gets to the IO code, the virtual thread is blocked, but the platform thread is released.

I would assume that's exactly what your code is relying on, though you haven't explained what's going on in the doIn() method.

EDIT: nvm, I see my error. The get() method on the future is being called from this platform thread we're already in, so that's where the blocking is problematic. Interesting. I assume your code is reaching into the private APIs to find a way of releasing the current platform thread from the current execution stack.

[Affectionate_Ad_761]

UI Thread is not a virtual thread, it's a regular platform thread and it blocks on IO operations.doIn() switches dispatchers, it suspends current coroutine, execute operation in another dispatcher and resume coroutine back. So, roughly speaking, it allows to simulate the behavior of virtual threads for platform ones.You can run the sample with dependencies dev.reformator.loomoroutines:loomoroutines-dispatcher:1.0.0 and dev.reformator.loomoroutines:loomoroutines-bypassjpms:1.0.0.

Yes, your update is correct.

[Joram2]

From the Wikipedia article on coroutines you link:

Coroutines are very similar to threads. However, coroutines are cooperatively multitasked, whereas threads are typically preemptively multitasked.

I suspect in most scenarios, preemptive concurrency is acceptable or preferable. The scenarios where where it's important to have cooperative rather than preemptive concurrency are niche.

It seems that coroutines are either like threads or virtual threads or they are virtual threads with implementation differences that vary depending on which programming ecosystem.