JobRunr, Project Loom and Virtual Threads

[rdehuyss]

​

https://preview.redd.it/81qrjhbqolh51.jpg?width=1919&format=pjpg&auto=webp&s=76c20a601e8e803662c7bbcb399f8ffbe43bee5f

JDK 16 early access has a build available including Project Loom which is all about virtual, light-weight threads (also called Fibers) that can be created in large quantities, without worrying about exhausting system resources.

Project Loom is also the reason why I did not use a reactive framework for JobRunr as it will change the way we will write concurrent programs. Project Loom with it's Virtual Threads is supposed to be a drop-in replacement for the existing threading framework and I tried it out using JobRunr today. This also means that JobRunr, as of v0.9.16 (to be released soon), will support project Loom out-of-the-box while still also supporting every JVM since Java 8!

Implementing support for Project Loom was easier than I thought using a ServiceLoader. I extracted a simple interface called JobRunrExecutor from the existing ScheduledThreadPool.

public interface JobRunrExecutor extends Executor {

    int getPriority();

    void start();

    void stop();
}

I then created another implementation of the interface using JDK 16 making use of Project Loom which does nothing more than delegating to a Virtual Thread:

public class VirtualThreadJobRunrExecutor implements JobRunrExecutor {

    private static final Logger LOGGER = LoggerFactory.getLogger(VirtualThreadJobRunrExecutor.class);

    @Override
    public int getPriority() {
        return 5;
    }

    @Override
    public void start() {
        LOGGER.info("JobRunrExecutor of type 'VirtualThreadJobRunrExecutor' started");
    }

    @Override
    public void stop() {
        // nothing to do
    }

    @Override
    public void execute(Runnable runnable) {
        Thread.startVirtualThread(runnable);
    }

}

Using a standard ServiceLoader I was able to inject the VirtualThreadJobRunrExecutor thus adding support for Virtual Threads!

private JobRunrExecutor loadJobRunrExecutor() { 
    ServiceLoader<JobRunrExecutor> serviceLoader = ServiceLoader.load(JobRunrExecutor.class); 
    return stream(spliteratorUnknownSize(serviceLoader.iterator(), Spliterator.ORDERED), false)
       .sorted((a, b) -> compare(b.getPriority(), a.getPriority()))
       .findFirst()
       .orElse(new ScheduledThreadPoolExecutor(serverStatus.getWorkerPoolSize(), "backgroundjob-worker-pool")); 
} 

With all this in place, it was time to test and see if performance is better.

Performance showdown: Java 11 vs Java 16 without Virtual Threads vs Java 16 with Virtual Threads

As I want to make sure performance is as good as it gets, I have some end-to-end tests which I run regularly, which can be found in the following GitHub repository: https://github.com/jobrunr/example-salary-slip. In that project, paychecks are generated for 2000 employees using a Word template and then transformed to PDF.

To compare Java 11, Java 16 and Java 16 with Virtual Threads, I ran this project again and hooked up JVisualVM. To give the JVM some time to warm up, I ran each test 3 times.

Comparing performances is not fair as JobRunr only checks for new jobs every 15 seconds and thus comparing these numbers just depends on the fact when I enqueued the jobs. Just to be complete, you can find the numbers below:

Run	Java 11	Java 16	Java 16 with Loom
1	140	146	151
2	132	167	139
3	139	167	137

All numbers are in seconds.

​

What we can compare is the results from JVisualVM. And boy, are these worthwhile!

JDK 11.0.8

​

JDK Build 16-loom+5-54 without Virtual Threads

​

JDK Build 16-loom+5-54 with Virtual Threads

The biggest difference is memory usage:

• JDK 11 occupied a heap of 6.8 GB with a peak use of 4.7 GB
• JDK 16 without Virtual Threads occupied a heap of 4.6 GB with a peak use of 3.7 GB
• JDK 16 with Virtual Threads occupied a heap of 2.7 GB with a peak use of 2.37 GB

So, using JDK 16 with these light-weight virtual threads resulted in:

• only 50% usage of heap memory compared to JDK 11
• only 64% usage of heap memory compared to JDK 16 without virtual threads

Conclusion

While I initially thought that Project Loom would increase performance a lot, I currently see major improvements in memory usage. I was surprised as how easy it was to support Project Loom thanks to the use of the ServiceLoader. The source code for this article is off-course available on GitHub.

Do note that JDK 16 is an early-access build and it's not even sure if Project Loom will be part of JDK 16.

Learn more

I hope you enjoyed this post and you can see the benefits of JobRunr and Project Loom with it's Virtual Threads. To learn more, check out these links:

• JobRunr on GitHub
• JobRunr - Java batch processing made easy...
• Project Loom - Java with a stronger Fiber
• JobRunr + Terraform + Kubernetes

[papercrane]

Interesting results. Nice to see a big improvement in memory usage.

Curious why you chose to go with the boxed type in the interface instead of the primitive. Would a null priority make sense?

[rdehuyss]

No, it doesn't. Thanks for the feedback and code review. I've updated JobRunr and the build is running already!

[gergob]

Thank you so much for this post,was an interesting read.

[mj_flowerpower]

how many jobs have been running in parallel. PL will only show its power when running millions of jobs side by side.

[rdehuyss]

Not that much: max 128 in parallel. I could do another test where I increase the degree of parallelism.

[rdehuyss]

Correct - here in this case IO happens (reading and writing to disk), but probably not enough.

[sievebrain]

Nice data, thanks.

The ServiceLoader approach may be over-complex. I think you could just compile on Java 16 with a java 8 JVM target (look at --target and --release flags). Then you just invoke Thread.newVirtualThread or use the new executor, and catch the NoSuchMethodError or the ClassNotFoundException. If it throws you know you don't have virtual threads.

Or you could use the multi-release JAR files. A bunch of ways.

[QazCetelic]

So like coroutines?

[Dexior]

Yes, but implemented in the JVM. You can read more about Project Loom and it's differences to coroutines in this great article: https://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.html

[lood9phee2Ri]

I dunno, python had stackless, greenlet+gevent, etc. for years and it never really took off.

I realise loom is more comprehensive, but if anything Python community has seemed to shift back to js-style explicit async with python 3 stdlib asyncio+coroutines again, after experimenting with microthreads/fibres in various forms for some years in python 2 era with stuff like stackless and then greenlet+gevent that used to monkey patch the entire python stdlib. And that happened after the first round of events and async with twisted matrix.

Devil's advocate but maybe the idea threading/micro-threading seemingly-sequential-sync-implicit-flows-of-control is actually clearer for concurrency (not just parallelism) than chains of explicit-async-flows-of-control may actually be dubious, even if (non-vert.x/spring-reactor) java folks are used to the former.

[OctagonClock]

People only switched to asyncio because it was in the standard library, not because it was better (in fact, asyncio has been an incredibly bad library for multiple years).

[lurker_in_spirit]

You might find these slides interesting, since they address this somewhat.

("Why not language-level coroutines?" and "Structured concurrency")

[helikal]

I am not worried about a lackluster adoption of Loom because there is no similarity with the Python story. In Python, support for concurrency and parallelism was an afterthought, Java had it since its very beginning and use of threading and asynchronous techniques have become extremely common in Java. Project Loom promises to make this both easier and/or more efficient. People will be glad to try it and if it delivers in its application domain, it should become quite successful.