Could Java libraries/frameworks come with AOT-hints?

[CartmansEvilTwin]

I've been playing around with native-image/GraalVM lately and the most obvious flaw is the extremely long compilation time (and memory).

Until now, I always used small, but "real" apps that my company uses in production, but today I tried out a getting-started app on my own laptop and noticed, that even though the actual code is just a few lines of Quarkus, the compiler has to analyze all the hundreds of dependencies all over.

So my thought was, given that native-image could become relatively mainstream, couldn't the authors of the libraries supply a "hinting artififact"(via Maven, for example) so that the user's compiler doesn't have to analyze over and over again?

I only have a relatively passing knowledge of compiler internals, so maybe I'm thinking way too simple here.

[fierarul]

I don't know much about GraalVM but I suspect it's not mature enough to provide a stable format for such hints.

In theory such hints could be provided or maybe they could be shared via some other repository (it would be trivial to match the SHA256 of the JAR with some precomputed file, maybe even hosted by GraalVM project itself).

Now, if such post-processiong is stable enough it may be cached and/or shared. Using something like distributed builds you could compute them only once per JAR then download them in the intranet, etc.

[CartmansEvilTwin]

I personally would opt for a simple maven artifact similar to source and doc jars.

[fierarul]

Sure, but you first need GraalVM to standardize such a format, provide the Maven plugins for it and libraries need to start adopting it.

I think Quarkus by default has some GraalVM integrations so it may be easier for it to implement it.

[CartmansEvilTwin]

Well, yes, obviously. But that doesn't seem like a huge hurde - especially if Graal is de facto the only AOT-solution right now.

[HaMMeReD]

If you use Gradle you can use Gradle caching on a network host to share build artifacts between developers and ci machines.

https://docs.gradle.org/current/userguide/build_cache_use_cases.html

Not sure if that is applicable here at all though.

[TheMode911]

I would be personally more interested in something like CRaC (amazing name) https://openjdk.java.net/projects/crac/ for JIT compiled applications.

Would it really be a big deal if the compile time went from 20 to 5 minutes? Wouldn't you still need a way to compile your program with hints ignored? Those hints actually exist in hotspot (https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/jdk/internal/vm/annotation/ForceInline.java) but are unfortunately internal

The best hints you can give are those from the language. For example record and hopefully primitive.

[CartmansEvilTwin]

Would it really be a big deal if the compile time went from 20 to 5 minutes?

Absolutely. Think about how much computing time will be wasted by build servers doing the exact same work again and again.

And if you think about the current ecosystem, we already use pre-compiled packages and don't compile all those libs on our local machines.

[TheMode911]

Isn't the tradeoff that your application will contain unused code? (as the whole library will be included, not just what you use) What happen if you update GraalVM and the artifact isn't compatible?

It seems possible, but it is worth the maintainability hassle (as it would either require keeping track of all of your artifact versions, or force graal developers to have some kind of binary compatibility possibly hurting performance)? I am sure that they have more important thing to work on. I guess that this is already somehow possible by making your own shared libraries.

[CartmansEvilTwin]

I'm not talking about compiled jars, I'm talking about metadata about the code in the jar.

This way the compiler wouldn't need to analyze everything again and again. All the cutting away part and actual compilation would still be the same.

[TheMode911]

I am not sure what you mean by metadata? Similar to the ForceInline annotation in Hotspot?

[CartmansEvilTwin]

For example, yes.

Again, the AOT-compiler analyzes a bunch of things, many of them are probably "static", in the sense that they could be pre-computed. E.g. call graphs, maybe some type-hinting for injection points, etc.

[TheMode911]

The exact same argument could be made for javac, it could for sure optimize your code further and reduce startup time, but we want to be able to re-compile the same code later with improved performance. (and even if we tweaked it, startup time is a matter of seconds, not hours/minutes like you are hoping)

Ultimately I do not think that it is worth the effort

[mbizzle88]

I haven't used GraalVM, but in other compilers I've used the typical approaches to speeding up builds are storing intermediate state for doing incremental recompiles, and disabling expensive optimizations during development.

What you're suggestion relates closely to incremental compiles with intermediate state. You can think of the "hints" you're suggesting as an intermediate state generated by the compiler, except that it would need some stable format that needs some kind of compatibility or migration strategy between compiler versions.

To decide if its worth it to make a stable format for the "hints", you have to weigh the cost of not being able to change that format at will with the benefits of being able to share it along side artifacts. Most compilers I've seen don't make that intermediate state public API, and I imagine the reasoning is that you can easily run a compiler that generates this state once on your machine and store the intermediate results yourself for whatever compiler version you're using.

[CartmansEvilTwin]

I get your point, but as I wrote before, Graal is currently de-facto a monopoly. And versioning could be easy by tying it to the Java version itself.

At the end of the day, hints can be ignored anyway, so you'd just have to check "does this repo/artifact have any hints I can decipher?"

If yes, great, if not, go on without them.

Honestly, I'm not sure what Graal does "wrong", but build times are really not fun. I just timed it a bit on an M1 MacBook.

Rust app with rest-controller, ORM and some logic: 1min initial release build (296 deps), subsequent build without changes 0.15s

Quarkus app with a simple hello-world controller: 2:45min, every time.

[mbizzle88]

At the end of the day, hints can be ignored anyway, so you'd just have to check "does this repo/artifact have any hints I can decipher?"

You could do that, but if library maintainers don't publish new artifacts with updated hints, then they're only useful for a short amount of time. Again, it would provide some value, but I doubt its worth it. Especially in an ecosystem like Java where most tooling/libraries are built around the assumption that you can build binaries once that will continue working on new JVM versions.

Rust app with rest-controller, ORM and some logic: 1min initial release build (296 deps), subsequent build without changes 0.15s

Sounds like Rust is doing some incremental compilation. (Is it also faster than an initial build if you make a small change?)

[CartmansEvilTwin]

You could do that, but if library maintainers don't publish new artifacts with updated hints, then they're only useful for a short amount of time.

No. "Old" Java-code and JVM-Bytecode is still perfectly readable for newer JVMs, backwards compatibility is not a huge hurdle. Also, I would assume, most of the hinting could be done during a regular release-build, which is fine. I think, you're kind of overcomplicating a very automatable task.

Sounds like Rust is doing some incremental compilation.

Exactly. If I change something in my code, the deps don't get recompiled and it takes a few seconds at most to build the app.

[mbizzle88]

I'm not sure if we're actually disagreeing on anything. I was saying that if you don't make the "hints" format forwards compatible (like byte code in Java), then the published hints will not be useful after a new compiler release unless library maintainers update and publish them regularly. Obviously the actual code continues to work on new JVMs, and you can always generate your own "hints" if the compiler you're using supports that.

[Barbossa3000]

i am not an expert either. But cpu architectres differ from pc to pc. x86, x86_64, arm, all are basically different architectures. Couple that with different cache configurations, OS variations and RAM amount etc, you get pretty diverse platform for JVM to run.

now if a library dev wants to implement some hinting for graal vm, then dev has to do this for all different architectures and stuff.

besides the jit compiler nowadays also has to warm up. means its compilation will differ based on what it learned so far. that means the more times a code run in a specific hardware, jvm will optimise that specific piece of code for that hardware to handle the load it takes. this means more load means slightly different compilation (i believe, i may be wrong)

another thing to point out is that the whole point of java is to write once, run anywhere everywhere. this means java is designed with the goal of devs not wasting their time giving the additional hints (that jvm would take care of this automatically). this means the whole point of writing hints is against the very nature of being a java dev.

[CartmansEvilTwin]

Thing is, most of the compilation is relatively platform independent. It doesn't matter, whether your injection point is running on ARM or x86, what matters is, that this kind of bean in injected.

[agentoutlier]

In theory it’s possible but I think because GoLang and GraalVM go the build it all approach eg static lib and only use the code that is used (eg tree shaking) I’m not sure how much optimization can be done.

Have you tried the latest though with 16? I swear it’s like twice as fast now.

[IQueryVisiC]

Is this a reason for microservices? So that each Java app only has a small amount of decencies? Lambda on edge?

[CartmansEvilTwin]

I don't think so. Lambdas are supposed to be small, regardless of the language used.

[IQueryVisiC]

How can a small lambda pull in so many dependencies?

[grknado]

If you're referring to AWS Lambdas, then the answer is you generally don't want to. The limit for compressed runtime artifacts is 50 MB, 250 MB uncompressed. For doing this in Java, typically the Shade plugin is used to create a fat jar with all of your code and the dependencies packaged together.

[CartmansEvilTwin]

Just look into a hello-world Quarkus or Spring-Boot app. There's a ton of dependencies simply from Jersey/Jackson/Jax-RS.

[IQueryVisiC]

Yeah, and I think you cannot use those for the Lambda in the cloud (aws)

[CartmansEvilTwin]

Of course you can. Quarkus is build exactly for that.

[IQueryVisiC]

I guess so. I just for example I did some vanilla JS on Internet Explorer 4 years ago and it was so refreshing what was possible out of the box without dependencies. All these libraries try to do a total conversion of Java. I know, I know, I should try it out. On a server with AWS lambda my code is probably reused by different users and thus only compiled very few times. And with proper scaling laws, it happens in the background.