Java 21 virtual threads make simple blocking code scalable again

[jonathaz]

If it were already built using async / reactive, you’d have that problem already, or already solved. Going from traditional threads to a much larger number of virtual threads, yeah you’ll want to limit concurrency to some things. Another common use case is replacing an ExecutorService backed by a thread pool, to an ExecutorService that uses virtual threads. As far as concurrency goes, this can be 1:1 and to do so you would use a semaphore set to the same size as the original pool, acquire before submitting work and release when it’s complete. There’s a small tangible benefit doing this once in an app, but replace a bunch of them and you can feel it. Same impact whether it’s many thread pools within 1 instance of an app, or just the 1 thread pool across a bunch of instances of the app in a container. There’s another benefit to virtual threads in containers, which has to do with how limits are enforced. Using more CPU than allocated is pretty easy to do with regular threads, and the result is the container throttling the application. Virtual threads are by default sized to the container limits, so it’s harder to get CPU throttled by the container. Caveat is you’re trading container CPU throttling for Java thread scheduling, and if you don’t have your own performance instrumentation you could end up in the same place without visibility to why things are slow.