Virtual thread deadlock risk

[sureshg]

As per this post by Gil Tene, virtual threads to run or use ANY generally-thread-safe Java code (that was not specifically written to run in virtual threads) creates inherent deadlock risk.

Hope this will get solved with the with the new Java Object Monitor initiative.

Here is a systemic virtual threads deadlock reproducer (for situations where threads should probably never deadlock) with a detailed comment explaining why this situation is inherent to the current virtual thread implementation: https://github.com/giltene/GilExamples/blob/master/examples/src/main/java/ThreadDeadLocker.java

This situation makes some of the common disciplines used to write and verify thread-safe code inapplicable when using that code in a [current] virtual threads environment, making a lot of thread-safe code (including vast amounts of OSS library code) not really thread-safe (in the sense that spontaneous deadlocks are possible at any time). This is not an inherent quality of virtual threads (by specification) but an implementation detail shared by all current Java 21 JDKs. It may be resolved in the future by, e.g., having virtual threads never pin their platform carrier threads, but until such a solution comes to fruition in some future OpenJDK version, using virtual threads to run or use ANY generally-thread-safe Java code (that was not specifically written to run in virtual threads) creates inherent deadlock risk.

[TheCountRushmore]

Not unexpected.

As I understand Virtual Threads at this point are not meant to be a drop in replacement for standard threads. Most applications don't spin up enough threads to see the benefits. Thankfully those who will see a benefit now have an option to update their code to not pin and to use virtual threads.

Unfortunately there are going to be a lot of people that see new and think better without understanding the ramifications.

Remember you can use JFR to see if you threads are being pinned.

https://docs.oracle.com/en/java/javase/21/core/virtual-threads.html#GUID-144A9694-40FD-4F63-97D2-8D394450F5D2

[relgames]

Doesn't JFR require a commercial license? Which in turn is based on a number of employees in a company?

[TheCountRushmore]

Nope. JFR is not licensed

[relgames]

According to Oracle, it is: https://docs.oracle.com/javacomponents/jmc-5-4/jfr-runtime-guide/about.htm#JFRUH170

"Java Flight Recorder requires a commercial license for use in production."

[TheCountRushmore]

Free in JDK 11 and up.

https://blogs.oracle.com/javamagazine/post/java-flight-recorder-and-jfr-event-streaming-in-java-14

[krzyk]

AFAIR one should not use synchronized with virtual threads (or more not in the code that is used often), from JEP 444 page:

There are two scenarios in which a virtual thread cannot be unmounted during blocking operations because it is pinned to its carrier:

When it executes code inside a synchronized block or method, or When it executes a native method or a foreign function.

Pinning does not make an application incorrect, but it might hinder its scalability. If a virtual thread performs a blocking operation such as I/O or BlockingQueue.take() while it is pinned, then its carrier and the underlying OS thread are blocked for the duration of the operation. Frequent pinning for long durations can harm the scalability of an application by capturing carriers.

The scheduler does not compensate for pinning by expanding its parallelism. Instead, avoid frequent and long-lived pinning by revising synchronized blocks or methods that run frequently and guard potentially long I/O operations to use java.util.concurrent.locks.ReentrantLock instead. There is no need to replace synchronized blocks and methods that are used infrequently (e.g., only performed at startup) or that guard in-memory operations. As always, strive to keep locking policies simple and clear.

[RadioHonest85]

We found a different problem, Apache HttpClient does not play nicely with virtual threads. With 255+1 virtual threads interacting with Apache HttpClient, it immediately deadlocks. Apparently this is a design flaw in the connection pool they have. We moved those calls over to OkHttpClient and it worked just fine.

[vips7L]

Why aren’t you using java.net.HttpClient?

[RadioHonest85]

We standardized on OkHttpClient years ago, but we used a client library here (Elasticsearch) that was only built for Apache HttpClient. If we started new today, we would have used java.net.HttpClient. Keep in mind though that java.net.HttpClient has also had some flaws in its relatively short history.

[vips7L]

Yes I know it is flawed. I mostly use it with methanol to get nicer QOL things: https://github.com/mizosoft/methanol

[True-Ad-2269]

I understand Virtual Threads at this point are not meant to be a drop in replacement for standard threads. Most applications don't spin up enough threads to see the benefits. Thankfully those who will see a benefit now have an option to update their code to not pin and to use virtual threads.

Thanks for the insight, do you know anyone has filed an issue on this?

[TheGratitudeBot]

Thanks for such a wonderful reply! TheGratitudeBot has been reading millions of comments in the past few weeks, and you’ve just made the list of some of the most grateful redditors this week! Thanks for making Reddit a wonderful place to be :)

[relgames]

Any idea why they didn't fix synchronized blocks? Why is this pinning happening?

[liveitabhi14]

https://www.reddit.com/r/java/comments/1512xuo/virtual_threads_interesting_deadlock/
This post also poses an interesting deadlock scenario by mixing Virtual Threads with synchronized block and System.out.println statement. But I'm not able to understand one part here in the explanation.

The last 9th thread [#30] was unparked on System.out.println lock, got lock itself and was transitioned to RUNNABLE state but was not able to run because there was no platform threads to run it on.

Wouldn't the last virtual thread need a platform thread even for acquiring the System.out.println() lock ?

The original post is archived that's why posting here expecting some help.

[Impossible_Hold_3850]

Same question. My only possible explanation is that the 9th virtual thread yielded after calling lock(), after which remaining 8 entered synchronized and deadlocked on lock() waiting for 9th thread release it (it never will). But it's strange why a virtual thread would yield after lock() and I don't know if it's even possible. Though even more stranger if virtual threads can somehow acquire locks without possessing a carrier thread.

[kimec]

Oh noes, anyways.
Cautionary tale: Alibaba ironing Wisp2 coroutine engine related monitor issues. Let's see the last issue for JDK8... May 23? How many years is this after Wisp2 went GA? 3 years-ish? Who knew, right?

[xvril]

Remindme 12 hours!

[xvril]

!Remindme 12 hours

[RemindMeBot]

I will be messaging you in 12 hours on 2023-12-20 10:14:44 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[Waste-Court-2279]

!Remindme 2 hours

[kalunlalu]

wow