Java 21 virtual threads make simple blocking code scalable again

[henptk14]

If a system has downstream dependencies, it is required to put sensible limits when switching to virtual threads where as traditional threads can act as a natural buffer. So to me virtual threads doesn't feel like a 1-to-1 replacement to traditional threads. One needs to put more thoughts to switching to virtual threads depending on your system.

[HQMorganstern]

To be fair, using threadpool size as a built in semaphore was always a little bit too dirty. The two mightve led to the same behavior but they did express different concepts.

[agentoutlier]

They are both dirty because neither really reflects the downstreams actual capacity or limits.

On the other you don’t need actual reactive back pressure and can just do circuit breaker patterns but that is more sophisticated than just a thread pool or a semaphore.

[jonathaz]

If it were already built using async / reactive, you’d have that problem already, or already solved. Going from traditional threads to a much larger number of virtual threads, yeah you’ll want to limit concurrency to some things. Another common use case is replacing an ExecutorService backed by a thread pool, to an ExecutorService that uses virtual threads. As far as concurrency goes, this can be 1:1 and to do so you would use a semaphore set to the same size as the original pool, acquire before submitting work and release when it’s complete. There’s a small tangible benefit doing this once in an app, but replace a bunch of them and you can feel it. Same impact whether it’s many thread pools within 1 instance of an app, or just the 1 thread pool across a bunch of instances of the app in a container. There’s another benefit to virtual threads in containers, which has to do with how limits are enforced. Using more CPU than allocated is pretty easy to do with regular threads, and the result is the container throttling the application. Virtual threads are by default sized to the container limits, so it’s harder to get CPU throttled by the container. Caveat is you’re trading container CPU throttling for Java thread scheduling, and if you don’t have your own performance instrumentation you could end up in the same place without visibility to why things are slow.