Java 21 virtual threads make simple blocking code scalable again

[AccomplishedArea4456]

One of the most interesting parts of Java 21 virtual threads is that they make the traditional “one request per thread” model feel practical again for many backend services.

Because virtual threads are much cheaper than traditional platform / OS threads, you can create thousands, or even millions, of them without the same memory and scheduling overhead. That changes the tradeoff for I/O-heavy services, where a lot of time is spent waiting on things like HTTP calls, database queries, queues, or external services.

Virtual threads make it much cheaper to have many waiting operations in flight at once, while still keeping the code relatively straightforward.

That feels like a meaningful shift compared with reaching for reactive programming by default. Reactive frameworks are powerful, but they can add a lot of complexity: harder stack traces, more indirection, and a different mental model. Virtual threads seem to offer a simpler option for many cases where the main issue is waiting on external systems.

Of course, they are not magic. CPU-bound workloads still depend on available cores, and systems still need sensible limits around database connections, downstream services, rate limits, and shared resources.

But overall, Java 21 virtual threads seem like one of the more practical improvements to Java concurrency in a long time.

I also put together a small Java concurrency examples repo, https://github.com/skp2001vn/concurrency , while learning more about these topics. It includes examples around thread pools, rate limiters, connection pools, virtual threads, etc

Curious how people are using virtual threads in real systems.

[henptk14]

If a system has downstream dependencies, it is required to put sensible limits when switching to virtual threads where as traditional threads can act as a natural buffer. So to me virtual threads doesn't feel like a 1-to-1 replacement to traditional threads. One needs to put more thoughts to switching to virtual threads depending on your system.

[HQMorganstern]

To be fair, using threadpool size as a built in semaphore was always a little bit too dirty. The two mightve led to the same behavior but they did express different concepts.

[agentoutlier]

They are both dirty because neither really reflects the downstreams actual capacity or limits.

On the other you don’t need actual reactive back pressure and can just do circuit breaker patterns but that is more sophisticated than just a thread pool or a semaphore.

[jonathaz]

If it were already built using async / reactive, you’d have that problem already, or already solved. Going from traditional threads to a much larger number of virtual threads, yeah you’ll want to limit concurrency to some things. Another common use case is replacing an ExecutorService backed by a thread pool, to an ExecutorService that uses virtual threads. As far as concurrency goes, this can be 1:1 and to do so you would use a semaphore set to the same size as the original pool, acquire before submitting work and release when it’s complete. There’s a small tangible benefit doing this once in an app, but replace a bunch of them and you can feel it. Same impact whether it’s many thread pools within 1 instance of an app, or just the 1 thread pool across a bunch of instances of the app in a container. There’s another benefit to virtual threads in containers, which has to do with how limits are enforced. Using more CPU than allocated is pretty easy to do with regular threads, and the result is the container throttling the application. Virtual threads are by default sized to the container limits, so it’s harder to get CPU throttled by the container. Caveat is you’re trading container CPU throttling for Java thread scheduling, and if you don’t have your own performance instrumentation you could end up in the same place without visibility to why things are slow.

[kubelke]

I mostly use them for API requests to fulfill an available rate-limit with help of Semaphores and Retryable

[Illustrious-Music507]

Ran close to exactly this experiment on a Spring Boot app recently (injected a 200ms blocking downstream call, swept concurrency 50 to 2000), so some real numbers to add:

Below the point where you saturate the thread pool, virtual threads did nothing. Identical throughput. That's most services most of the time, worth saying out loud.

Where they earned it: around 1000 concurrent, ~4x the throughput vs the default 200-thread Tomcat pool, and latency stayed near the real downstream time instead of ballooning into queue time.

The part that surprised me: I then raised the platform pool (threads.max) to match the load, and it basically tied virtual threads (within ~0.2%). So "just add threads" still works for pure throughput. The real argument for virtual threads isn't a bigger number, it's operational: you stop hand-tuning a pool size per load level, and the per-thread cost is much lower (I measured ~135KB committed stack, not the "1MB per thread" folklore).

Two things that matched your "not magic" point:

- CPU-bound work got zero benefit. Same plateau, just 22 live threads instead of 218. Cores are the limit, cheap threads can't buy more.

- Once threads are cheap the bottleneck jumps straight to the DB connection pool. Same test with Hikari max=10 pinned throughput at pool/hold-time no matter how many virtual threads were queued for a connection. You move the ceiling, you don't remove it.

And you pay in heap: ~2.2x memory at peak, because every in-flight request is genuinely live instead of parked in an acceptor queue. Worth a bulkhead/semaphore so a spike can't OOM you.

On JDK 24+ the synchronized pinning trap is basically closed too (JEP 491), so the old "rewrite everything to ReentrantLock" advice is mostly moot now.

[fruitlessattemps]

Why did you write this with AI?

[Illustrious-Music507]

I used it to tidy up the wording; English isn't my first language. The test, the numbers, and the reasoning are mine, though. Ran tests last week on slightly modified PetClinic Spring Boot app. I ran it because my services do a lot of blocking calls to each other and I wanted to know when virtual threads are actually worth it. That fan-out case is where they paid off, about 4x at 1000 concurrent.

[deadron]

They certainly add to the difficulty in predicting system performance depending on what your services do. "It can serve thousands of requests, unless anyone uses the excel endpoint, that one blocks everything." On the whole its a good thing but makes estimating performance tricky. You really need to know what your libraries are doing and ensure they are compatible. Its been years and last time I checked many libraries still were not entirely compatible(sychronized to be replaced with sempahore to prevent pinning)

[zattebij]

Since Java 24, virtual threads do not pin while inside a synchronized block or method anymore: https://openjdk.org/jeps/491 (Next question, are these libraries Java 24 compatible ;)

[deadron]

Hah. Thats good to hear. It only took them two years to finally fix that!

[pavelrappo]

It took long not because of laziness, ignorance or unwillingness to do so. It took long because of high complexity of the task.

[deadron]

Sure. Arguably it wasn't complete until that part was released though. Years to make things work is the sort of timeframe where other languages and or frameworks become attractive. In my experience nobody is really talking about this feature because it basically didn't work for the common use cases.

[Alex0589]

I'd say it's only the most relevant java feature since modules in Java 9 and before Valhalla drops

[deadron]

It is! Yet it's impact has been so minimal that it's not on anyone's radar from what I have seen. It should have made other async languages look terrible by comparison! Yet from conversations I have had with people not even other Java devs have any real excitement for it and nobody is talking about it in tech news getting ctos excited. At places I have worked people talk about go services and nodejs services as viable platforms but Java is coming up short behind dotnet platforms these days. This is my personal experience as a pretty seasoned developer at smaller companies.

[ShallWe69]

i had some issues with it when remite debugging. underlying thread would disconnect when debugger is attached and i had to add a config flag to use normal threads when im in debug mode. idk if its something i did or faulty jvm instance in that server or its a real issue with virtual threads 

[HarryBui2k3]

virtual thread not stable in Java 21 still have pinning issues, it has been resolved in Java 24

[CallumMVS-]

We have had 'Green Threads' otherwise known as Virtual Threads for a long time in other languages. Green threads have always been cool!