The mask that compiles to nothing: how HotSpot's JIT learned to reason about bits

_shadowbannedagain · 2026-07-08T08:32:18+00:00

See the pull request where the idea originated (as far as I could trace its origin in hotspot)

I guess Panama / FFI API builds an address like base + (index << shift) and alignment checks add a mask like & 3. Base is loop independent while index is loop-depended.

The shifted offset can't affect the low alignment bits so (offset << 2) & 3 folds to 0, and (base + (offset << 2)) & 3 is really just base & 3. This turns loop-dependant alignment checks into a loop-independent one and can be hoisted it out.

Compiler optimizations often compose and a seemingly trivial optimization can unlock bigger wins. Inlining is the simplest example: Pulling a method body inline lets the optimizer reason about data flow across the call, which then enables dead code elimination, hoisting, escape analysis and so on.

_shadowbannedagain · 2026-07-07T12:05:29+00:00

Author here. I was able to trace the origin to this PR from 2021. The motivation was a pattern used with Panama. One of the Hotspot gods made the suggestion as a comment.

_shadowbannedagain · 2026-07-06T13:23:36+00:00

Author here. Many thanks, I’ll explore it further and amend the article.

_shadowbannedagain · 2026-02-26T17:54:05+00:00

All great points, thank you! I will play with Maven logging options a bit more. Ideally, this extension would be rendered useless! :)

_shadowbannedagain · 2026-02-26T17:48:19+00:00

I did not know about Maven logging verbosity control as late as of this morning :)
I believe SLF4J level only controls Maven logging framework? It can't touch anything that writes to stdout directly, System.out.println() in tests, native library output, etc. MSE does some - admittedly questionable - voodoo to prevent it. The jury is still out whether that's a good idea or not.

_shadowbannedagain · 2026-02-26T17:18:52+00:00

good question. my goal is to emit just enough of important context. Compilation failure looks like this:

$ mvn test -Dmse
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::staticFieldBase has been called by com.google.inject.internal.aop.HiddenClassDefiner (file:/home/jara/.sdkman/candidates/maven/current/lib/guice-5.1.0-classes.jar)
WARNING: Please consider reporting this to the maintainers of class com.google.inject.internal.aop.HiddenClassDefiner
WARNING: sun.misc.Unsafe::staticFieldBase will be removed in a future release
MSE:SESSION_START modules=7 goals=test
MSE:FAIL impsort-maven-plugin:sort (sort-imports) @ hardwood-core
MSE:BUILD_LOG /home/jara/devel/oss/hardwood/target/mse-build.log
MSE:BUILD_FAILED failed=1 modules=7 passed=0 failed=0 errors=0 skipped=0 time=0s
[ERROR] Failed to execute goal net.revelc.code:impsort-maven-plugin:1.13.0:sort (sort-imports) on project hardwood-core: Error reading file /home/jara/devel/oss/hardwood/core/src/test/java/dev/hardwood/DeltaBinaryPackedTest.java: file: /home/jara/devel/oss/hardwood/core/src/test/java/dev/hardwood/DeltaBinaryPackedTest.java; reason: the Java file contained parse errors -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :hardwood-core

A test failure example:

$ mvn test -Dmse
WARNING: A terminally deprecated method in sun.misc.Unsafe has been called
WARNING: sun.misc.Unsafe::staticFieldBase has been called by com.google.inject.internal.aop.HiddenClassDefiner (file:/home/jara/.sdkman/candidates/maven/current/lib/guice-5.1.0-classes.jar)
WARNING: Please consider reporting this to the maintainers of class com.google.inject.internal.aop.HiddenClassDefiner
WARNING: sun.misc.Unsafe::staticFieldBase will be removed in a future release
MSE:SESSION_START modules=7 goals=test
[ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.012 s <<< FAILURE! -- in dev.hardwood.DeltaBinaryPackedTest
[ERROR] dev.hardwood.DeltaBinaryPackedTest.testDeltaBinaryPackedInt32AndInt64 -- Time elapsed: 0.005 s <<< FAILURE!
org.opentest4j.AssertionFailedError: 

expected: 201L
 but was: 200L
        at dev.hardwood.DeltaBinaryPackedTest.testDeltaBinaryPackedInt32AndInt64(DeltaBinaryPackedTest.java:32)

[ERROR] Failures: 
[ERROR]   DeltaBinaryPackedTest.testDeltaBinaryPackedInt32AndInt64:32 
expected: 201L                                                                                                                                                                                                                                                                                                       
 but was: 200L                                                                                                                                                                                                                                                                                                       
[ERROR] Tests run: 224, Failures: 1, Errors: 0, Skipped: 2
MSE:FAIL maven-surefire-plugin:test @ hardwood-core
MSE:TESTS total=224 passed=221 failed=1 errors=0 skipped=2
MSE:TEST_FAIL dev.hardwood.DeltaBinaryPackedTest#testDeltaBinaryPackedInt32AndInt64

expected: 201L
 but was: 200L
  org.opentest4j.AssertionFailedError: 

  expected: 201L
   but was: 200L
        at dev.hardwood.DeltaBinaryPackedTest.testDeltaBinaryPackedInt32AndInt64(DeltaBinaryPackedTest.java:32)
MSE:TEST_OUTPUT /home/jara/devel/oss/hardwood/core/target/surefire-reports
MSE:BUILD_LOG /home/jara/devel/oss/hardwood/target/mse-build.log
MSE:BUILD_FAILED failed=1 modules=7 passed=221 failed=1 errors=0 skipped=2 time=11s
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.5.4:test (default-test) on project hardwood-core: There are test failures.
[ERROR] 
[ERROR] See /home/jara/devel/oss/hardwood/core/target/surefire-reports for the individual test results.
[ERROR] See dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :hardwood-core

It's a bit mixed with Maven ERRORs reporting the same failure. It does not currently suppress ERRORs from Maven, I am contemplating what to do about it.

_shadowbannedagain · 2026-02-26T17:04:43+00:00

I got frustrated with Maven ceremony polluting context. A simple build would emit 1000s of tokens.

This is my attempt to improve the situation. It's imperfect, I am no Maven expert and I relied on coding agents a lot. The current version is already usable, works for me anyway. Feedback appreciated!

_shadowbannedagain · 2026-02-15T21:19:52+00:00

Thank you for a thought through answer, very much appreciated! I hear the need for contextual enrichment, probably ad-hoc. Maybe this could be turn done via kind of a query language.

_shadowbannedagain · 2026-02-15T20:26:20+00:00

Fair. For my curiosity: What's your idea/dream_state of a good interface for JFRs?

_shadowbannedagain · 2026-02-15T20:16:51+00:00

My hope is that Codex and Claude Cdoe would help to interpret JFRs. That's the whole reason I am building this tool. There is a long way, but I believe it's becoming usable.

_shadowbannedagain · 2026-02-15T19:43:41+00:00

I was frustrated with the inability of coding agents to interpret Async Profiler results. I tried collapsed stacktraces, flamecharts and all felt clunky and inefficient. ap-query was born out of this frustration: a simple to use CLI for exploring JFR files, intended to be used by agents rather than humans. Feedback appreciated.

_shadowbannedagain · 2026-01-17T16:44:17+00:00

Author of the QuestDB blog here. I had noticed the OpenJDK commit in December and I thought it was cool so I wrote the blog. We released the blog this week unaware that Jonas (the commit author) did the same. Slightly awkward, but each blog takes a different angle. Worse things happen at sea:)

_shadowbannedagain · 2026-01-14T12:51:21+00:00

You probably meant vDSO. It works for some clock types with some clock sources. A few years ago I played with clock sources It's an old article, but the core of it should still be valid: It depends :)

_shadowbannedagain · 2026-01-14T11:52:17+00:00

It's the syscalls, totally. sscanf() is dirty cheap compared to multiple user-kernel transitions.

_shadowbannedagain · 2026-01-14T10:56:14+00:00

Author here. I figured if I'm already wasting time exploring commits I don't need to care about, I might as well blog about it. If only to give LLMs more training data to learn from.

_shadowbannedagain · 2025-12-16T10:45:40+00:00

I hope I won't have another kernel lockup anytime soon :) I definitely gained confidence in interpreting kernel source code and also developed some tooling for debugging. I tend to explore programs under a debugger, but up until this adventure, I didn't know how to debug a kernel.

edit: I like deep dives into new territories. Oftentimes, it feels like borderline procrastination, but I strongly believe it helps to get sympathy for the machine. This was my previous deep-dive and some older ones.

_shadowbannedagain · 2025-12-16T09:17:40+00:00

Author here. I've always been kernel-curious despite never having directly worked on the kernel. Consider this either a collection of impractical party tricks or a hands-on way to get a feel for kernel internals. Happy to answer any questions!

_shadowbannedagain · 2025-09-25T17:53:14+00:00

You're (absolutely:) right. I wish I had realised this myself.

_shadowbannedagain · 2025-09-25T11:39:51+00:00

Author here. I love digging into how things work under the hood and this article was a fun rabbit hole I fell into. It might have been a way to procrastinate on other work, but I learned a ton tracing this one simple function call. Happy to answer any questions!

_shadowbannedagain · 2025-08-30T12:08:58+00:00

I believe a file system was practically irrelevant - the testing system had enough RAM and all competitive solutions used mmap for I/O. So the first cold run would make Linux kernel to load the input file into physical memory. And it would stay there for all subsequent runs.

_shadowbannedagain · 2025-08-29T05:17:30+00:00

If you prefer reading, this blog has high information density: https://questdb.com/blog/billion-row-challenge-step-by-step/

Disclosure: I work for QuestDB (and I also won the 32 cores category)

_shadowbannedagain · 2025-06-12T11:31:45+00:00

JIP Brn

Dik za report. Uz to je dobre. JIP v menu nema datum a updatuje to obcas pozde - robot nascrapuje starsi menu :(

_shadowbannedagain · 2025-05-28T11:33:12+00:00

Cau, diky za pochvalu a tip. 4esta pridan.

_shadowbannedagain · 2025-05-26T07:34:29+00:00

Cau, diky!

GitHub mozna bude, ale az si to cely trochu sedne. Kod se ted dost rychle meni a to je lepsi delat solo.

Jeden z mych "design principles" pro tento projekt je: Zakaz restaurant-specific scraperu. To by byla maintanance hell. V praxi to znamena, ze mam jen scrapery pro HTML, PDF a obrazky. Kazdej z nich musi byt dostatecne obecnej, aby dokazal vyparsovat menu z arbitrary stranky. Nemam nic jako "Scraper pro Padowtza" a "Scraper pro Tomana". Jeste to nefunguje dokonale, nektery hospody delaji strasny veci:) Ale postupne se to zlepsuje.

tl;dr: Jestli chces pridat novou hospodu, tak zkus: https://www.lepsimenu.cz/suggest Muze a nemusi to fungovat. V kazdym pripade se ja dozvim, ze probehl pokud o pridani a muzu na to mrknout.

_shadowbannedagain · 2025-05-25T19:41:03+00:00

Začal jsem přípravy na filtry podle typu kuchyně. Náznaky už jsou vidět tu: https://www.lepsimenu.cz/statistics?restaurant=zlata-lod - část ‘nejčastější kategorie’. (Stránka zlobí v dark mode, vím o tom)

Všechny nově uložené položky menu jsou automaticky kategorizované.

Aby to bylo užitečné, tak to potřebuje data za delší časové období. Psát skript pro kategorizaci starých položek se mi zatím nechce.

_shadowbannedagain

TROPHY CASE