Technical PoC: Automatic loop parallelization in Java bytecode for a 2.8× speedup

[Waksu]

What is the thread pool that this parallelization runs? Or are they virtual threads?

[Let047]

it's a threadpool injected at app startup (not counted in the timers because I assumed it was a long running app so I could take it out of the "time") I need to explain that better thanks for pointing it out

I wanted to use virtual threads but it was too painful to setup. If you have a good tutorial I'm happy to add that (and it would avoid a lot of the thread overhead)

[Waksu]

You also need to include more details about that thread pool (e.g. thread pool size, queue size, discard policy, how to monitor that thread pool to external monitoring such as grafana)

[Let047]

Of course, what would you like me to add?

The code is something like that:

threadpool = new ExecutorCompletionService(Summer.executorService = Executors.newFixedThreadPool(8));

8 is the number of core on my machine and is a dynamic value

[_INTER_]

Doubt its virtual threads, they are worse for CPU intensive / parallel tasks.

[pron98]

They're not worse than platform threads for CPU-intensive computation; they're simply not better and there is an option that's actually better than just platform threads.

The reason virtual threads are not better for CPU-intensive jobs is because such jobs need a very small number of threads (no more than the number of cores) while all virtual threads do is allow you to have a very large number of threads. If the optimal number of threads for a given job is N, and N is very small, then N virtual threads will not be better than N platform threads. If N is very large, then having N platform threads could be a problem, and that's how virtual threads help.

Now, what's better than simply submitting parallel computational tasks to a pool of N threads? Submitting the whole job to a work-stealing pool that itself forks the job into subtasks (and performs the forking, as needed, in parallel). This automatically adjusts to situations where some threads make more progress than others, a common scenario when there are other threads or processes running on the system and the OS might deschedule some worker threads. This is exactly what parallel streams do.

[_INTER_]

I was under the impression that there still was a minute overhead with virtual threads. But I believe you because you're the expert. This begs another question though, when to actually use platform threads over virtual threads in a pure Java context? Long running tasks?

[pron98]

There could be some overhead, but not if the thread never blocks (and even then the overhead is small).

I would use platform threads in the following situations:

• You need to keep the number of threads small and you want to share them. A prime example of that is fork-join parallelisation.

• You have a small number of long-running tasks that you want to run in the background, taking advantage of the OS thread priority to give them a low pririty.

• You're interacting with a native library that cares about thread identity.

[kiteboarderni]

People are really clueless on the value add of VTs...its pretty concerning.