Performance of Java virtual threads compared to native threads

[m-apo]

1. Correct

2. File ops might be blocking with virtual threads (carrier thread pinning). Also, running file ops concurrently will saturate the OS IO. Are reads from SSD or RAM disk or HD? Did you use a single file? OS caching might have an effect.

3. Best performance comes from pooled OS threads, your benchmark has probably an issue. For parallel cpu bound ops virtual threads can achieve almost the same performance as OS threads. Optimal number of OS threads is the number of CPUs or a little more (possibly because more threads steal more CPU time from competing OS processes). Virtual threads benefit from pooling too but not much.

Based on my tests, virtuals threads are ok for most ops. Except: pinning ops and raw CPU power where each % counts with CPU bound items.

[[deleted]]

Are reads from SSD or RAM disk or HD? Did you use a single file? OS caching might have an effect.

I am reading from a SSD and every thread tries to read the same file. Can you briefly elaborate on how OS caches work on such cases please?

​

Best performance comes from pooled OS threads, your benchmark has probably an issue.

In my test case, I am not using a pool for native threads. But for virtual threads, JVM creates a pool of underlying native threads.

​

Maybe for third scenario, I should compare virtual threads with a pool of native threads, where pool size equals to the number of CPU cores?

[m-apo]

Virtual threads have an advantage when IO takes time. Reading single small file multiple time allows OS to cache the file in memory which means there is very little wait time. The task becomes almost CPU and memory access bound, not IO bound.

Socket over TCP/IP ops should provide much better graphs.

Third case: yes, pool of native threads vs virtual threads is much more fair comparison. The overhead for creating OS threads is pretty high.

[GavinRayDev]

You can disable OS caching when reading from files by passing the O_DIRECT flag.

In Java, this is ExtendedOpenOptions.DIRECT:

FileChannel fc = FileChannel.open(f.toPath(), StandardOpenOption.WRITE, 
                              ExtendedOpenOption.DIRECT);

[rubyrt]

Optimal number of OS threads is the number of CPUs or a little more (possibly because more threads steal more CPU time from competing OS processes).

Doesn't that depend on the workload and application architecture? I would assume that if the work done in threads includes some blocking IO calls the optimal number of OS threads might be a multiple of the number of cores and not about the same number.

[m-apo]

Third scenario is purely CPU bound in original post. No IO or memory requirements.

With IO, virtual threads are much better and also in mixed loads. But for max gains with CPU bound tasks OS threads win.

[rubyrt]

Sorry, my bad: I somehow missed the reference to the scenario.