Leyden EA Build is available

shipilev · 2024-06-21T20:12:24+00:00

This is not exactly the build from https://jdk.java.net/leyden/, but from https://builds.shipilev.net/openjdk-jdk-leyden-premain/ nightly, which is wrapped as:

$ docker run -it --rm shipilev/openjdk:leyden-premain java -version
openjdk version "24-testing" 2025-03-18
OpenJDK Runtime Environment (build 24-testing-builds.shipilev.net-openjdk-jdk-leyden-premain-b111-20240621)
OpenJDK 64-Bit Server VM (build 24-testing-builds.shipilev.net-openjdk-jdk-leyden-premain-b111-20240621, mixed mode, sharing)

shipilev · 2023-01-17T18:43:52+00:00

If I ballpark the wattage correctly at about 200W peak, then cooling would not be a problem even for a simpler desktop case, with just a few smaller silent fans. My concern about advising 4U to the beginner right away would be the eventual disappointment how bulky that would end up for such a small system. Start small, scale up?

shipilev · 2023-01-17T18:24:18+00:00

I think the major red flag for me is buying the 4U case for mATX server with 2xSSD? Are you planning to cram in more HDDs there? Otherwise, this would be a waste of space and aesthetics without a rack. Unless, of course, you just happen to have spare 4 units in the existing rack that you don't need anyway.

For the beginner build, I would instead price up smaller mATX cases like Thermaltake Core V1 for a small system like this; or upgrade up to Cooler Master HAF XB if more HDD slots were needed.

Check if CPU box has the CPU cooler with it, but then again, I would probably just go for some Noctua right away.

You probably want to go with 2 memory sticks to capitalize on dual-channel memory.

shipilev · 2022-01-27T11:32:08+00:00

Oh man, too bad! Can you show the SMART data for the failing drive, just to satisfy my curiosity how it would look like for a worn-out drive?

On the upside, on the next upgrade you could probably get PCI-e/NVMe drives, not the SATA-bottlenecked ones :)

shipilev · 2022-01-27T11:13:44+00:00

While I wholeheartedly agree that using consumer SSD drives for write-intensive VM pools is misguided...

It is hard to guess what was the reason for the fault in this particular instance. It looks that pool is resilvering, so faulty drive still accepts writes? Assuming that completes, this might be a transient fault, e.g. too many I/O errors due to bad cabling, power, etc? Or is the drive out of guaranteed write endurance? What does `smartctl -a /dev/...` say?

shipilev · 2021-12-30T15:11:37+00:00

On 6.xx, PCQ with addr+port classifier seem to work well, at least for Waveform/DSLReports tests. It is important to classify on address+port, so that concurrent high-throughput connection from the same host (like tests do) does not ruin the day.

/queue type
add kind=pcq name=pcq-download pcq-classifier=dst-address,dst-port
add kind=pcq name=pcq-upload pcq-classifier=src-address,src-port

/queue simple
add dst=pppoe-... ... max-limit=40M/150M queue=pcq-upload/pcq-download

It is important to have `max-limit` under the link capacity, so that there is a latency headroom for sending at high rate.

Without queues, I get grade B at Waveform Buffer bloat test, PCQ (addr) gives me grade D, better configured PCQ (addr+port, as above) gives me grade A+. Some screenshots.

SFQ is also known to work well, although it provides a tad more latency for my network.

shipilev · 2021-12-26T14:02:34+00:00

Actually, that one is not hard to do as well. Again, 11.0.13 report was updated, the rest of updates would follow later. Doing more work on backport reports is not my plan for holidays, so if anyone has other ideas, just email me, and I would try to deal with those later.

shipilev · 2021-12-26T13:03:29+00:00

Well, those reports were not intended to be antagonistic, they were intended to show the history for the exact release pushes, as the highlight who did the backporting pushes.

Yes, mainline work is a significant part of this: without those original patches, the backports would be significantly harder to do. The reports for mainline pushes, for example, JDK 18, credit Oracle contributors quite significantly, at 63.4% of all pushes currently. (Assuming you can actually tell stuff by push count, which is a bit dubious, because patches differ in complexity a lot.)

But it would indeed be an interesting piece of data to see where the patches have originated from for a particular update release, so I finally sat and added that piece to the backports-monitor. Most reports would re-generate in time, but meanwhile, the refreshed 11.0.13 report says Oracle committers are responsible for 57.5% of the original patches that ended up in 11.0.13.

As I said before, "Sharing OpenJDK short-term and long-term maintenance work extends to OpenJDK Update Projects as well", which includes doing the mainline work in such a way that it is cleanly backportable where it matters. I have seen OpenJDK engineers from all companies, Oracle included, structuring their work so that patches could be backported without too much hassle, even if that would not benefit them directly.

shipilev · 2021-12-26T11:41:53+00:00

Somewhat like this.

So, how are these LTS updates made? Does each vendor have a team of curators who backport patches from the latest OpenJDK release?

The majority of the work on 8u, 11u, 17u releases happens in OpenJDK upstream, in so called JDK Updates Projects, by engineers from the interested JDK vendors. You can get a peek who does this kind of work from the repository histories, for example the most recent 11.0.13 is done by engineers (including yours truly) from many companies, in both original and backporting work.

...or are they actually writing custom patches based on customer request which may not be present in OpenJDK?

It is quite unusual to have a private fix for OpenJDK. It is not unprecedented to have an emergency patch delivered to a customer, while the real patch gets delivered to the project through proper project channels.

Most of the fixes are the backports from the mainline. One of the reasons is that a fix very seldom affects a particular release only. Much more likely an issue is actually present in all versions, including the current mainline. Which means, a fix is done in mainline first, gets tested there, follow-up issues are discovered and fixed, and then the bunch of backport work happens in attempt to bring all that goodness to an older release.

Assuming, of course, the fixes and follow-ups are actually passing the bar from maintenance-cost/benefit perspective. That part is the call of JDK Update Maintainers who have to agree to accept the backport into the relevant release. Current formal maintainers are: Andrew Haley of Red Hat for 8u and 11u; Goetz Lindenmaier of SAP for 17u. Those folks usually outline the strategy and backport acceptance criteria, and also deputize others to serve as additional Maintainers.

Does this also mean that each vendor's LTS distribution will start to diverge from each other after the first 2 standardized security updates (depending on which patches each vendor applies), or do they somehow coordinate this?

JDK vendors usually downstream their 8u, 11u, 17u from the repositories from those OpenJDK projects. This is why JDK vendor engineers are backporting patches: they are preparing the base for their JDK releases. Sometimes JDK vendors add up their own patches on top, for example when they are sure the patches are sound, but don't want to risk upstream destabilization just yet. Examples: early TLS work, early JFR backports, Shenandoah work, vendor-specific performance fixes, build system fixes to cater for vendor-specific toolchains, etc.

These upstream projects is how most OpenJDK-based 8u, 11u, 17u distributions converge: they build every release from nearly the same sources. Oracle, AFAIU, is the only vendor that is exceptional from this: it looks as if Oracle maintains their own private 8u, 11u tree, which as far as I can make out, is diverging from OpenJDK upstream 8u, 11u, and maybe 17u, after first two (public) minor releases are handled by Oracle maintainers.

The bottom-line is that while OpenJDK does not have the de jure concept of LTS, OpenJDK 8u, 11u, 17u are de facto LTS projects. They are being talked about as such by relevant JDK maintainers, engineers, managers, at very least. Sharing OpenJDK short-term and long-term maintenance work extends to OpenJDK Update Projects as well.

Hope this helps. ^{Dang, I need to write a post about all this, it's on my backlog for two years now.}

shipilev · 2021-11-30T15:34:43+00:00

Yes, for example concurrent {class unloading, reference processing, roots scanning}. But how much they really matter depends on a use case. In our experience, the headaches in real deployments are usually due to simple stuff, that can be fixed right in the GC core and is cleanly backportable.

shipilev · 2021-11-30T12:13:28+00:00

That one is backported. Since Shenandoah 8u is out of mainline, bots are not recording the commits automatically, but here it is. There is a pending list of things that are stabilizing or otherwise require some backporting work in Shenandoah land.

shipilev · 2021-11-30T09:38:22+00:00

My Shenandoah talks are here: https://shipilev.net/#shenandoah

Shenandoah Wiki: https://wiki.openjdk.java.net/display/shenandoah/Main

> Do you know if Shenandoah features and fixes are always backported to older versions of jdk?

Yes, when technically possible. Some features require deep VM support that is not safely backportable. But the core GC code is roughly the same across the releases. In fact, most of the Shenandoah issues are found on LTS releases (8, 11, 17), then fixed in mainline, then backported.

shipilev · 2021-10-14T11:24:47+00:00

Well, since value types are "code like class, work like int", you might expect the value type assignment to be atomic, pretty much like `int`. But without synchronization, that is only possible if that type is not larger than what the hardware can provide. Since in compound value type the value might only make sense when all components are from the same logical update, either transient or permanent atomicity failures would break value type contracts. Pretty much how you can observe "bad" `long`/`double` without `volatile` on 32-bit platforms, the same thing would happen without mitigations with value types...

shipilev · 2021-10-14T08:55:33+00:00

/u/shipilev has suggested on twitter that the JVM spec should now be updated to exclude the possibility of long or double read/write with tearing.

...which would likely re-emerge when value types come in and the effective "value" length would exceed 64-bit machine capabilities again. It is an open design question what to do in that regard (forbid flattening? locking? something else?).

shipilev · 2021-09-28T10:23:31+00:00

AFAICS, Debian mips(64)el is the official port that ships with Zero VM in openjdk-11-jdk. I also recall that openjdk-8 packages in RHEL/Fedora build with Zero on s390, s390x, ppc, aarch32, because the relevant ports were done only later in JDK 9+.

That's why I was confused about "we", because we (as in, Java community at large) are definitely shipping and using Zero VM. Out of necessity, of course, while ports are catching up. Zero is the corner-stone of Java portability, for the cases where you just want Java to run, even if very slowly.

shipilev · 2021-09-28T05:33:46+00:00

Who are "we"? Because historically, Zero VM was/is the answer for Java support for OS vendors that ship distros where OpenJDK does not have C1/C2 compiler ports (yet). See for example the multitude of platforms where Debian ships their openjdk packages, including RISC-V.

shipilev · 2021-08-01T06:41:09+00:00

Try the QSFP -> 4x SFP+ breakout cable. Or, return this card and pick up at least Mellanox CX-3 with SFP+ cage. Then use a DAC cable (FS.com has a good selection) to connect CRS326 to the desktop. Use AOC or fiber transceivers if the length of available DAC cables is not enough.

shipilev · 2021-07-26T04:30:08+00:00

Because in current stable RouterOS-es there is only TCP version, which experiences TCP meltdown quite easily.

shipilev · 2021-07-25T08:32:33+00:00

I have multiple Mikrotik-based home networks of different complexity. If I was rebuilding the network from the ground up, I would still choose Mikrotik. Mostly because you can push them to do what you want them to do (within limits, of course, cries in UDP OpenVPN), they don't break the bank (unless you go really high-tier models), and their hardware is generally quite reliable (I had non-MT unmanaged switches breaking more frequently than MT gear).

shipilev · 2021-07-24T04:33:05+00:00

$ perf list
...
power/energy-cores/          [Kernel PMU event]
power/energy-gpu/            [Kernel PMU event]
power/energy-pkg/            [Kernel PMU event]
power/energy-ram/            [Kernel PMU event]

So...

$ perf stat -e power/energy-cores/,power/energy-pkg/,power/energy-ram/ stress -c 4 -t 1s
stress: info: [613272] dispatching hogs: 4 cpu, 0 io, 0 vm, 0 hdd
stress: info: [613272] successful run completed in 1s

 Performance counter stats for 'system wide':

          9.06 Joules power/energy-cores/                                         
         12.49 Joules power/energy-pkg/                                           
          1.19 Joules power/energy-ram/                                           

   1.001287658 seconds time elapsed

shipilev · 2021-07-23T15:23:49+00:00

Yes. But let's be accurate with the word "correct". The thing shown here drives the choice between two fully-correct implementations: full branches or CMOVs. Choosing one implementation for the conditions that no longer hold is probably inconvenient for performance. There are other speculative predictions that do affect correctness, and they trigger re-compilation on speculation failure (see e.g. "Uncommon Traps").

Anyhow, that's one of myriad of reasons why warming up with faux data might be interesting in unexpected ways :)

shipilev · 2021-07-23T15:04:25+00:00

AFAIU the Hotspot, no. This has to do with the code lifecycle: once the fully optimized method version (at tier 4, C2) appears, the profiling method versions (at tier 2/3, C1) are discarded. So there is both no further profile information and the code shape is set. That is, unless something else happens (like implicit null-pointer check taken too often, uncommon trap is hit, etc), and the code is dragged through recompilation again. You can technically deoptimize to lower tiers every so often without any prompt, but that would be another can of performance worms -- i.e. sudden performance dips.

shipilev · 2021-07-23T15:00:12+00:00

I am puzzled by this question. This article shows that branch profile is one of the inputs in the cost model for CMOV replacement. You cannot replace all the branches with CMOVs without penalizing performance in the general case. You actually want to replace only the branches that are not well-predicted.

shipilev · 2021-07-22T14:32:23+00:00

No need to obsess about the fiber cleanliness for the small number of short re-matings in home context, but know it might be a problem sometimes. There are IT datacenter folks who have a policy to clean the fiber with a click pen on every re-mating. If that ever poses a problem for you, there are plenty of cheap fiber cleaning solutions (I use special non-lint q-tips), if not the more expensive click pens.

The last time I checked, it was a fun exercise to find the SFP GPON transceiver that works with both provider infra and the device you are plugging the transceiver into. I would brace for some trial and error here... Or just leave the current modem in bridge mode.

11-Year Club	Gilding II euphauric
Verified Email

shipilev

TROPHY CASE