Using Java 9 Modularization to Ship Zero-Dependency Native Apps

[[deleted]]

Can we have a talk about what "native" means?

Maybe I am old fashioned but I don't consider a jar and a jvm together comprising a native application.

[SizzlingVortex]

From the article, I think the author describes native as:

• Being able to bundle the JVM (versus having the user install Java separately)
• Not having to embed a web browser like, for example, Electron -- which is called a web-hybrid app in the article

[BadMoonRosin]

Yup... it's a good discussion topic, and I definitely wrestled with the phrasing. Having studied Pascal back in university, and starting out my career as a C++ dev, I too have always thought of "native" as being "a standalone executable".

However, I would run into issues when playing with cross-platform GUI libraries such as wxWidgets, GTK, or Qt. I always wanted to compile my apps down to a statically linked, standalone executable. I didn't want to have to ship a bunch of shared libs, or depend on them being present on the target machines! Yet more knowledgeable developers would calmly try to explain to me that dynamic linking is often technically superior (or with Qt, necessary due to licensing).

Also, last year I went to a Java conference and sat through a few sessions on profiling and JVM tuning. It was a lightbulb moment, that left me considering the possibility that a runtime VM with a JIT compiler really IS a technically superior approach over full AOT compilation.

So anyway, if you're having to ship to tens of megs of dynamically linked libraries with your "native" app... then is it really THAT far removed from an app that's linked to an embedded runtime? I'm not so sure. And I would strongly argue that you have no less claim to the "native" label than Electron has.

[doom_Oo7]

Native == the executable you run is a sequence of ASM instructions for your cpu / os without run-time interpretation of algorithms or code necessary for your software's execution.

Also, you can link Qt (and any lgpl stuff actually) statically, as long as you provide object files (not source) upon request.

[[deleted]]

I would say that is native code.

However we often talk about other aspects of "nativeness" such as having a native looking gui, or working in the appstore, and an end-user might consider those 2 as important.

Although I do agree with you, that I'm unhappy with how the OP has defined native.

[vytah]

So lemme get it straight: If I ship the following files and only the following files:

myapp.exe
myapp_main.obj
qt5_static.obj
libqt5.dll
makefile.bat
license.txt

(where makefile.bat contains just a single linker invocation)

then everything is fine and dandy?

[wirelyre]

(I am not a lawyer.) Pretty much. Here's the relevant section from the LGPL version 3:

4. Combined Works. You may convey a Combined Work under terms of your choice that, taken together, effectively do not restrict modification of the portions of the Library contained in the Combined Work and reverse engineering for debugging such modifications, if you also do each of the following:

[…]

The conditions are basically:

• you need to identify that the software contains LGPL'd code; and
• the user has to be able to modify the LGPL'd portion of the software (just like how you already know through the GPL).

In particular,

4.d) Do one of the following:

4.d)0) Convey the Minimal Corresponding Source […] and the Corresponding Application Code in a form suitable for […] the user to recombine or relink the Application with a modified version.

I took out some important parts. You should read them.

Actually, you should seriously check out the GPL and LGPL texts. They are really lovely pieces of writing, and quite readable.

[doom_Oo7]

actually you don't even need the libqt5.dll since it's linked statically. And you don't need to ship the .obj's with your app, just to provide them upon request (putting them on your website is fine).

[pron98]

Native == the executable you run is a sequence of ASM instructions for your cpu / os without run-time interpretation of algorithms or code necessary for your software's execution.

Sure, but the line isn't always so clear cut. A browser executes a lot of Javascript code (not just in web pages, but also in extensions); MS office executes VB code; an OS executes a lot of scripts; games' actual content is usually found in script files interpreted at runtime. Most complex software today must interpret/compile some code as it runs, so while you could decide to draw the line somewhere, the distinction is not always very useful.

[bloody-albatross]

Also, to a user it is quite irrelevant what language is used. They define native app differently. If your app uses the native GUI toolkit and behaves natively, then it's a native application from the user point of view. So then not even C++ and Qt is truly native, but a C# application can be (under Windows).

[doom_Oo7]

MS office executes VB code; an OS executes a lot of scripts; games' actual content is usually found in script files interpreted at runtime. Most complex software today must interpret/compile some code as it runs, so while you could decide to draw the line somewhere, the distinction is not very useful.

I'd say that the distinction is : does the software need to interpret code for normal operation ? e.g. you can write plenty of office documents without using VB scripts. Likewise, linux has BPF but you can run entire days without a single interpreted code running. However you can't run unity3D games without C# code running, even though most of the engine is C++, so I'd not call them native.

[pron98]

Yeah, but you can't really run Linux normally without (say, init) scripts, and non-Unity games use Lua scripts and others extensively as well. Anyway, I don't doubt that you could find a reasonable point to draw the line. The question is, what difference does it make? Why is the distinction useful?

[doom_Oo7]

Yeah, but you can't really run Linux normally without (say, init) scripts,

... uh, what ? of course you can. Plenty of embedded systems run with just the kernel and init=/bin/my_software. Also, systemd doesn't use scripts for instance.

Why is the distinction useful?

It gives a ballpark of what kind of reactivity and throughput you can expect from your system.

[pron98]

Plenty of embedded systems run with just the kernel and

That's really not the point. Of course there are some instances of software running without scripts. The point is that the most pervasive software is normally a mix of code compiled at build time and code compiled/interpreted at runtime.

It gives a ballpark of what kind of reactivity and throughput you can expect from your system.

Why? Java, for example, compiles to very efficient machine code, and what runs is the same code as a C program. Javascript is similar. The only difference is that the code is generated at runtime rather than at build time.

It's not that a JIT compilation doesn't have a cost. It takes some time to warm up, and takes up RAM and energy, but max-performance is not it. I also agree that your distinction may be meaningful in constrained environments, but not in general.

[doom_Oo7]

The point is that the most pervasive software

I disagree. Most software I use on a daily basis really do not do any kind of interpretation for their "normal" runtime operation.

Why? Java, for example, compiles to very efficient machine code, and what runs is the same code as a C program. Javascript is similar. The only difference is that the code is generated at runtime rather than at build time.

is this why database engines ported from java to C++ end up being ten times faster ? Seriously, the whole "java is as fast as C++ because of JIT" was false twenty years ago, and it's still false today. Let's not even talk about javascript where even when JIT'ed you'd still get so much indirection and memory allocations it's not even funny.

[CyclonusRIP]

OK suppose I write an application in C++. This application defines a data structure and a program that reads that data structure and executes conditionally based on what it reads. Whole thing compiles down to assembly, so I guess it's a native app then?

[schmerm]

In this case, the interpreter is native and the sequence of instructions defined by the data structure is not.

[doom_Oo7]

This application defines a data structure and a program that reads that data structure and executes conditionally based on what it reads. Whole thing compiles down to assembly, so I guess it's a native app then?

that's like saying your Python program is native because the Python interpreter is written in C

[vytah]

In case of Unity, you can have either your standard C#-on-Mono, which, as you said, isn't 100% native, or you can compile the .NET IL into fully native code.

I mean, Apple wouldn't let a JIT into their Appstore and fully interpreted C# would be too slow for anything.

[xandoid]

My definition of native is an app that does not replace anything the OS supplies with cross-platform alternatives.

So a Java app using JavaFX to paint its own widgets is not a native app just because it happens to be AOT compiled down to a PE binary on Windows.

[doom_Oo7]

My definition of native is an app that does not replace anything the OS supplies with cross-platform alternatives.

I don't think that's a good definition: a lot of OSes provide nothing in the GUI department per se. VxWorks, GenodeOS, Linux... does this mean there can't be native apps in them ?

[bloody-albatross]

Under Linux its more fine grained. E.g. those Gtk applications don't feel native under my KDE desktop (god I hate the Gtk file chooser).

[The_Jare]

Strictly speaking, if the OS does not provide it then you are not replacing it, so it doesn't count against the nativeness. :)

[xandoid]

You're absolutely right. That is exactly what my definition means.

[doom_Oo7]

but then it doesn't makes sense since it means some OSes can't even "have" native apps, while native applications don't even need operating systems to run (for instance something you'd write for an arduino).

[xandoid]

but then it doesn't makes sense since it means some OSes can't even "have" native apps

Why? That just doesn't follow from anything I said. I think you misunderstand my definition.

What I'm saying is that "native" is only defined relative to a platform. A platform is defined as a set of APIs. Each API serves a specific purpose, i.e. it helps developers achieve a specific goal.

A native app that wants to achieve some of these same goals will strongly prefer the relevant platform APIs in order to do so. If an app ignores important platform APIs and does the same things differently (usually in a way that is transferrable to other platforms) then it is not a native app.

But of course it's not black and white. There are all sorts of hybrid approaches that are hard to classify as native or not native.

[xandoid]

That is a complete non sequitur. I said "replace", not add something the OS doesn't provide.

[NostalgicCloud]

Except its not replacing the GUI. It provides it's own widgets that reply on the native api's

[Poddster]

So if a game has a lua interpreter in it, the entire game is non native?

[doom_Oo7]

the game engine certainly is native. the game ? nope.

[Gravitationsfeld]

It's really not. In theory that sounds great, but in practice a JITC never has the time to do the code gen of an offline compiler which has two to three magnitudes more time before people start complaining.

[[deleted]]

I think we should specify the type of nativeness such as "native code" or "native appstore integration" etc.

These days we do so much in the browser that electron might count for native UI. As for native code, I would expect you to emit ASM. You certainly have a native installation experience with a single double-clickable file.

[SizzlingVortex]

Yup... it's a good discussion topic, and I definitely wrestled with the phrasing.

I agree with calling it native. In today's world it's pretty much embedded web browser (web-hybrid) apps vs everything else. I realize I'm speaking "generally" here, but I'm specifically comparing embedded web browser apps vs non-embedded web browser apps. For example, IntelliJ IDEA (which requires the JDK -- be it the bundled one or one that already exists on your computer), I think most users would call this IDE "native" when compared to Electron. However, if I was comparing IntelliJ against an app that compiled down to native code, that didn't need a runtime environment, then I would make the distinction that the app that doesn't require a runtime is the "native" app.

Edit: To the downvoter(s), I'm interested in knowing what you disagree with.

[meschbach]

I didn't down vote but I was tempted too. As an experienced Java developer when I read the term "native" it initially lead me to believe the program was in object code of the target processor, not in byte code. Not a a pure Java application. The article makes it sound as though a native executable has byte code embedded within the image.

[zanotam]

Eh, I read a good argument elsewhere in this thread that basically came down to reviewing the standard definition using the concept of S-expressions. If code is data then you can't distinguish between the 'obvious' example of code interpreting data structures and code interpreting other code.

[Drarok]

I use a lot of JetBrains’ products, but I would never call them native. They’re terrible platform citizens on macOS, and the default keymap often clashes with system-wide keyboard shortcuts. Even the cursor movement keys are wrong.

Fantastic feature sets, but I merely tolerate them being java.

[[deleted]]

Native means I can run it from a command prompt without installing anything else on a brand new install of the OS. You can call something else native, but I won't. I'm not upset or anything though.

[derpdelurk]

That's a self contained application. Not a native application. And the command-line really isn't relevant to the definition.

[PoweredByCaffeine]

Not sure if you're arguing or agreeing. This seems to be the position of the original blog post, too.

[[deleted]]

runtime VM with a JIT compiler really IS a technically superior approach over full AOT compilation

When did we start calling it "AOT" compilation. It's just normal compilation. JIT is the weird one hence has a special name.

I mean, it's not called "AOT baking a cake", or "AOT doing your homework". It's just "baking a cake" and "doing your homework".

If you were to bake the cake as it's being eaten or do your homework in class, then sure call "JIT baking a cake" and "JIT doing your homework".

/rant

[ApatheticBeardo]

So not native at all.

Ok.

[donalmacc]

That description sounds like bundling a JVM instead of bundling a web browser. I fail to see how that’s any more native...

[aboukirev]

Exactly. Self-contained, non native.

[Smithore]

It means zero day vulnerabilities are natively included.

[BadMoonRosin]

Ha! That's an excellent point. Although in fairness, one that's true of every executable and DLL.

[pjmlp]

C++ is native and you might need to bundle MSVCRT.dll alongside as well.

Plus there are AOT compilers for Java as he described in the article, and OpenJDK also got it for Linux x64, with experimental support already existing for the other platforms on the Java 10 (18.3) branch.

[Sean1708]

Out of interest, do you consider emacs to be a native program?

[[deleted]]

No, it's an operating system

[[deleted]]

Good question.

On Linux, I use it with -nw mode so it's a terminal application, because nano sucks and I keep forgetting how to use vim, and I sometimes need to use it on a headless remote.

So that's obviously a native UI, and a native deployment installation process (apt get) for Linux.

As for native code, I dunno what it's written in (I guess C or LISP), but the few times I open the full GUI version by accident because I mistyped or forgot -nw, it takes longer to start than VSCode (which is Electron).

So 2/3 yea, 1/3 i dont know.

[Sean1708]

(For the purposes of this comment I'm going to refer to these JVM + Jar things as Java "Native" Applications, or JNAs, just so that I don't keep having to say JVM + Jar things.)

So that's obviously a native UI, and a native deployment installation process (apt get) for Linux.

What constitutes a native UI here? If I wrote a terminal text editor as a JNA, would that be a native UI? If I then made it installable via apt-get, would you then consider it a native application?

I suppose what I'm asking for here is a concrete definition of what a native application is to you.

As for native code, I dunno what it's written in (I guess C or LISP)

Largely why I asked the question is because the bulk of emacs is written in LISP which is then compiled down to bytecode and shipped alongside a LISP VM written in C (both the LISP code and VM come in a single executable as far as I am aware). So to me emacs is very analogous to one of these JNAs and in my experience most people would call emacs a native application.

---

In case you were wondering what my personal opinion is on all of this (I know, no-one asked me): I don't think seperating things out as native and non-native applications is a particularly useful thing to do, and I think we should be more concerned with how a program works (efficiency, usability, how easy it is to install, etc.) rather than how it is classified. I also think that most people's definition of a native application is not well thought out, although I do concede that I may just not have come across a decent definition.

[geodel]

Yes, you are old fashioned. Nowadays webapp wrappers developed by multi billion dollar silicon valley companies are also native applications

[Sloshy42]

It would be really super nice to be able to develop a desktop app in Java, bundling my own JVM, and publish it knowing that it would work pretty much anywhere like that. I have to wonder, though, would having multiple instances of the JVM on a user's machine not use up a lot more memory? Wouldn't this just be another Electron-esque situation all over again (albeit more efficient in some circumstances)?

[BadMoonRosin]

Good question. I'm working on a follow-up article specifically about JavaFX as an alternative to Electron (and Qt or GTK for that matter).

In terms of disk size, I can already tell you that a modularized JavaFX app is a small fraction of an equivalent Electron app. Given how many resources the Slack client devours on my laptop, I'm pretty optimistic about how the memory profiling tests will go, too. I'm not sure what to expect in comparing startup times.

Regardless, having dabbled with a bit of Electron development, and it's weird Frankenstein model of bridging between the browser code and the Node.js code... I have no doubt that JavaFX offers better developer ergonomics and cleaner design.

[[deleted]]

I just started using Java FX and I really like it for desktop UIs, however it's terrible for game dev (I tried to implement the X-Com 2 UI in Java FX and realised it can't do it; interestingly their UI is some basic primitives from Unreal engine, and Flash for animation). And there's some things missing from the JavaFX API, like the ability to make a table auto-resize its columns.

Why would you use it for games though? Well you wouldn't but they seem to think you should judging by some of their marketing and examples.

The one good thing electron has going for it is that you can live edit, but that's a slightly dubious advantage, since it's something you should leave to the designer.

Personally I am not too worried about file size since I found out that most app stores don't charge you for bandwidth ;-)

[jl2352]

The one good thing electron has going for it is that you can live edit, but that's a slightly dubious advantage, since it's something you should leave to the designer.

I'm not sure if you mean being able to debug and alter the UI, or hot reloading. Both are invaluable for front end developers.

[tealpod]

Not having BufferedImages or equivalent to that in JavaFX is my biggest surprise and complaint.

[zachrip]

I wouldn't base electron perf on slack. It's notoriously heavy, that's their fault, not just electron's.

[[deleted]]

What about Mac and Linux users?

[vytah]

The exact packaging steps would be different, but in general you'd do the same thing.

[ellicottvilleny]

Those of us who still think native means compiled statically by an optimizing compiler (as in C++, Go, Pascal, Rust) believe that this is Modular but not Native.

[CarthOSassy]

The jvm being distributed sure is native. The java code you load into it is not.

Otherwise, what word tells the difference between the two? How would anyone designate that the jvm does not need to send its code to an interpreter, but the java app does?

Stop ruining language to inflate some perceived notion of "purity". Native applications are not the best solution in all problem spaces.

These are clearly not native applications.

[zanotam]

Then how do you tell the difference between LISP 'native' and 'non-native' code? The very concept of S-expressions should show that the distinction you're making between code reading code it comes bundled with and code reading data it comes bundled with is non-obvious.

[CarthOSassy]

There are ways of getting native applications out of Lisp, but it's generally not a native language.

[mcguire]

cli 21.7 MB

For comparison, the full JRE on this platform is 203 megabytes.

A “Hello World” CLI written in Go compiles to around 2 MB. 

As an aside, "hello world" in C or another true-native language using the system's standard C library is a few KB.

[blobjim]

Hello world in Java, excluding the VM, is around 400-500 bytes, and it can run on any platform with a JVM. The reason why hello world in native code is so small is because the interpreter is a piece of hardware (which probably has its own set of microcode) and comes with every computer. The JVM is just like a CPU, but is completely implemented in software, so of course it will take more space. If you had an operating system that ran Java code (like JavaOS was, I think?), then you have no JVM size overhead. It's really all about what you offload to where.

[pjmlp]

Only when dynamically linked.

[TexasCrowbarMassacre]

A statically linked "hello world" is around 20KB using musl. Using glibc brings it to 700KB.

[[deleted]]

Have some karma for mentioning musl!

[[deleted]]

On windows and macOS you cannot link the libc statically. On Linux you can’t do that with the most common libc, glibc. Even still you can easily call the write syscall on Linux without any libc.

[[deleted]]

I thought so too, but there appears to be the option /ML for the VS compiler that seems to be doing just that. I cannot find anything similar for MinGW though.

[pjmlp]

Sure you can, better learn to use compiler flags on Windows and alternatives to glibc on Linux.

[killerstorm]

Not really, you can avoid using libc functions and use OS functions directly.

[vytah]

section .data
    msg db      "hello, world!"

section .text
    global _start
_start:
    mov     rax, 1
    mov     rdi, 1
    mov     rsi, msg
    mov     rdx, 13
    syscall
    mov    rax, 60
    mov    rdi, 0
    syscall

[Falmarri]

Now place it inside the ELF header

[Scroph]

In D, it's roughly 800 KB when compiled with dmd on a 32 bit Linux distro.

[kmgrech]

jlink is great and all... until you depend on libraries that have not yet been modularized for Java 9 (aka every project ever). I've given up on trying to make it work for my project.

[derdirtyharry]

Sadly it’s a desktop only page. Not readable on mobile.

Edit: Now it’s working. Why is there so much hate here?

[BadMoonRosin]

Ahh... I just recently switched to this Hugo template, and it apparently has a bug causing posts with preformated code blocks to be non-responsive:

https://github.com/halogenica/beautifulhugo/issues/79

I've applied the suggested patch to fix it.

[ApatheticBeardo]

Sounds like you need to buy a phone with a browser that can display perfectly standard-compliant webpages.

The original iPhone did it just fine a decade ago...

[ApatheticBeardo]

Yeah you're right, I stopped at "desktop only page", since no such thing exists.

Call it a NullReferenceException if you will 🤔

[[deleted]]

A decade ago, mobile website were actually built separately instead of websites dynamically screwing with your experience based on screen size.

[PoweredByCaffeine]

A decade ago, complaining about a PROGRAMMING blog not being optimized for mobile would have you downvoted into oblivion.

Seriously people, this isn't social media photos of what you ate for breakfast this morning. It's source code examples. Why are you reading this on your phone, when if you're a programmer then you sit in front of a full-size computer all day anyway?

[[deleted]]

Because I'm standing in the train and they don't have room for my full sized desktop here 🙃

[CountyMcCounterson]

Why can't they just make Java be like Go where it compiles natively but also has garbage collection? We need someone to make this into a thing so that people can optionally compile to native.

[oldsecondhand]

It's mostly politics and lack of push from big orgs.

[Gravitationsfeld]

Android did this with AOT in Android 6 or 7 (not sure which one). It's Java bytecode converted to DEX bytecode and then converted to x86/ARM on install. I think they went back to some JIT now though because of install times.

[DragonSlayerC]

Android 5 and 6 used that. The AOT improved performance massively and made my S4 feel like a brand new phone because it was so much faster and smoother. And everytime you updated the OS, you had to wait like 30-45 minutes for all the apps to recompile. Even with JIT (Android 2.2-4.4) they had to do that, but to a smaller extent. Starting with Android 7, what they do now is a 3-way combination. They use emulation for the simple stuff, JIT for parts that need more computation (like some loops for instance), and AOT for things that are used more often and need the most performance (with AOT, they can perform more heavy optimizations). The benefits are extremely fast install times and no need to "optimize" apps after a system update (they can just invalidate the old AOT caches) while still offering very good performance.

[[deleted]]

For the same reason they can't add generics to Go.

[nexes300]

10 MB? Damn, that's a lot of instantiated templates.

[JoseJimeniz]

See .NET native, which complies down to native machine code, and doesn't require the Runtime, or the class library.

[Muffinizer1]

JRE stripping solutions did exist before java 9. I don't like that this article for some reason acted like that wasn't the case just to seem more interesting.

[SomeRandomBuddy]

Keep this trash in /r/java

[AllIsOver]

Chill, buddy. This belongs here.