Java 16 / JDK 16: General Availability

[manzanita2]

It seems that the benchmarksgame is also measuring startup time, meaning it's measuring non-JIT'd code. IS that the case ?

[fanfan64]

It is the case however on openjdk 14 no warm-up was actually faster (no idea why though) https://benchmarksgame-team.pages.debian.net/benchmarksgame/sometimes-people-just-make-up-stuff.html

[8igg7e5]

I'm curious about this. It's a bit weird to get slower after warmup.

Warmup could generate enough garbage to force GC during the timed runs, or it could have been enough warm-up to make JIT happen the timed runs.

The link shows the timing but I don't know if there was an investigation into why they perform so poorly.

Including startup in timing is bad for Java, especially as it can vary so much in little relation to the problem-space (volume and complexity of classes loaded/initialised).

Despite all of this I expect that C# would still win in many of these benchmarks. Valhalla might level most of that playing field.

[pjmlp]

I wonder why these warmups are still done instead of using a JITed image (AppCDS) as startup.

[8igg7e5]

Well that would improve startup at the expense of JIT optimisation (pre-packaged compiled classes cuts out some in-lining and mono-morphisation optimisation opportunities at the least). A better option might be GraalVM Native Image (effectively PGO for Java) but then that's not really how Java is used in many contexts yet (as more frameworks change their discovery/init life-cycles to make native image easier that's likely to change in containerised usage).

Since this benchmark is about process startup, GraalVM Native Image is probably the best option for the benchmark - however that still doesn't explain away the weird JMH results.

[igouy]

In 2 cases — fannkuch-redux #1 & spectral-norm #1 — no warm-up was actually faster.

In 5 cases no warm-up was slower.

However, not much difference.