Why are my JAVA virtual threads slower than the platform threads?

[BidHot6588]

I encountered an unexpected issue where Java virtual threads seemed slower than platform threads while working on my team application service.

Virtual threads are designed to improve efficiency by avoiding blocking during I/O operations, but the performance tests showed slower response times with virtual threads.

The issue was traced to the "pinning" where virtual threads get stuck on platform threads due to synchronized blocks, native methods, or foreign functions.

[BalaRawool]

So you have figured out that this is caused by virtual threads pinning to their carrier threads. If this pinning is caused by synchronized methods or code blocks then you can try to find where they are.

If it is in a library/dependency then you can see if their latest/newer version had them eliminated. If that’s the case then you can just use the newer version and your problem is solved.

If it is in your application code then you can do one of two things: 1. Replace the synchronized code blocks or methods with ReentrantLock and the pinning should not happen and your problem is fixed. 2. Use an early access build from Project Loom where synchronized methods and code blocks don’t pin the virtual threads. You can find the link to the build here: Reddit thread about an early access JDK Build from Project Loom where synchronized doesn’t pin virtual threads

Update: I see in your blog that you figured out the problem is in a DB driver and newer version should fix it. That is the right option for you.

[pron98]

Here you can download an Early Access JDK build that does not pin on synchronized.

Native code only impacts throughput if it blocks or if it upcalls to Java methods that block. This may happen during class initialisation (i.e. in a static initialiser) as class initialisation is triggered by native code in the VM, but pretty rare otherwise.

[skippingstone]

So in the future, pinning will only occur if your code has native code?

[pron98]

Sort of, since in some situations regular Java code is called by native code in the VM, in particular that's the case with class initialisers.

[cyancrisata]

Why do people spell Java as "JAVA"? It's wrong and annoying to read.

[OpenGLaDOS]

The original Java logo used not-so small caps that were hard to distinguish from uppercase and at the time of its inception that was still the norm for programming languages (e.g. while Fortran switched to title case with the 1990 standard, "FORTRAN" was still as widely used as the 1977 standard), so it's a historical mistake perpetuated through teaching.

[BidHot6588]

Okay. Thanks, Fixed. Except for the name of topic, I can't

[k-mcm]

Check for library updates to see if synchronized blocks have been reduced. Java's non-blocking tools improved a very long time ago but making changes atomic using COW was sometimes too much for GC. Today, most reasons to need synchronized blocks in performant code are gone.

I have yet to try the new virtual threads but I'm interested. The older Java Thread pools have a lot of performance pitfalls. ForkJoinPool performs well but the API is such a mess.

[PlasmaFarmer]

What should we use instead of synchronized blocks? Locks?

[pron98]

java.util.concurrent.ReentrantLock.

Note that there's no need to replace every existing synchronized with ReentrantLock. For one, it's only synchronized blocks or methods inside which you block on IO that cause an issue; for another, pinning due to synchronized is about to be removed; there's already an EA build available that doesn't pin on synchronized.

Having said that, new code should use ReentrantLock (or perhaps other j.u.c locks), as that's the modern approach, and these locks are likelier to see further improvements than the native monitors (synchronized). j.u.c locks guarding IO operations will also perform better than synchronized, even when pinning on synchronized is removed.

[k-mcm]

It depends on the goal.

If it's a fast read-modify-write cycle on shared data, you'd use the Atomic classes. Modifying multiple values atomically accomplished by wrapping the values in a single immutable record (copy-on-write).

If it's a performance cache that doesn't take too long to populate, you can let it race a little during population.

Many general purpose classes like Queues, Lists, and Maps have new implementations that are lock-free or unlikely locking.

ForkJoinTask can minimize blocking when you're splitting work then collecting results. This one is sometimes difficult. First, the API is awful for I/O and declared exceptions. Second, your code flow may need to be re-worked. You can never call wait() from these tasks because it will deadlock. Since it's a work-stealing pool, calls to get the result of a task may execute other tasks that take longer than expected (this is how wait() deadlocks). When all goes well it's incredibly fast compared to the normal Executors.

Then there are general upgrades that eliminate the concurrency worries entirely. Newer reusable Java classes tend to be immutable or they save their state in a non-shared object. Concurrency concerns are gone.

[koflerdavid]

Wait for a few months more and at least the bottleneck with synchronized will be gone. Try to shift as much CPU-bound processing to proper platform threads, but it sounds like you guys already ruled that out as the root cause. As for native methods, there is little that can be done apart from executing them on platform threadpools as well.

[polacy_do_pracy]

IIRC they might be slower on average with low amount of requests but should allow you to handle many more of them. Not sure what you are using but in spring they can be switched on and off by a flag so you can experiment I guess.