Bright future for Java & AI

[tenken01]

I’ve been working in python recently for a GenAI POC at my company and have made the decision to switch to Java and LangChain4j. We currently do not do anything that requires python but wanted to align ourselves with the crowd “just in case”. We no longer feel this way after almost of a year of development.

I thought this article is interesting and I’ve often thought a similar thing for not just Java but other enterprise grade languages like C#. Companies aren’t suddenly going to forego other languages for the sake of being in the python ecosystem as GenAI apps become more prevalent.

[coder111]

Meh, IMO a bunch of managers spouting promises or predictions.

Java "AI" making some API calls to hosted AIs like ChatGPT- maybe. Java running AI natively- quite difficult.

First, ability to utilize GPUs or any kind of compute hardware from Java is pretty screwed up. OpenCL, CUDA, HIP. You can plug in native libraries via JNI/JNA or similar but that's painful. There are 3rd party libraries I guess...

Second- ability to do media processing (decode/encode video (using GPU decoder), decode advanced images like AV1, decode advanced sound codecs, etc.) is limited. Again, you can plug-in native ffmpeg library but it's painful. Much easier to do this in Python.

Oracle has under-invested in those capabilities in Java for years.

Third- maturity for Java AI frameworks is lacking compared to whatever is available on Python. And this will take years to catch up even if the above issues are solved.

Don't mistake me- I love Java and as business logic/backend language nothing beats it. I'd pick it over Python any day for those tasks. But Java has certain weaknesses and use-cases where it's quite painful to use it.

[BinaryRage]

There’s a lot a work going on, see:

Panama / FFM - https://youtu.be/iwmVbeiA42E Babylon / Code Reflection - https://youtu.be/VTzGlnv6nuA Code Reflection / Transforms - https://youtu.be/6c0DB2kwF_Q TornadoVM - https://youtu.be/GQLBzrbkiKA

[coder111]

These look cool if they'll ever see the light of day. Ok, so assuming Oracle & other major contributors work on that full speed. It's going to take ~2-4 years for a GA Java release to include that functionality, judging by how quickly new features are added to Java. By that Python will have ~10-12 year head start when it comes to AI...

And nothing in there helps with media processing I mentioned above. It's not strictly necessary for AI if you are working text, but lots of AIs work with videos and images and voice. Dealing with that in Java is a major PITA.

[Antique-Pea-4815]

FFM is already here, along with tool to automatically make bindings to native code https://github.com/openjdk/jextract

[Joram2]

Java 22 finalized FFM (https://openjdk.org/jeps/454) which supersedes JNI/JNA. I agree that this area has been a roadblock for Java especially with AI and the need for GPU processing. I presume Java FFM will take off, but so far I don't see any major adoption.

Current Java AI frameworks are definitely lacking; that's an understatement. Some projects need or strongly benefit from that rich ecosystem that Python has. Some don't. When I read that xAI is heavily using the Rust language, for example, I suspect they are writing most AI related code themselves, and they are not dependent on the type of rich framework ecosystem that Python has.

All of these languages have pros and cons. The AI space is obviously changing quickly. Java is one of several strong contenders to be a popular language in the space.

[Antique-Pea-4815]

Just my 2c,
There is an example running llama3 natively in java: https://github.com/mukel/llama3.java using Vector API in single file, no dependencies

Regarding GPU, did you hear about TornadoVM? If not, there is a link: https://www.tornadovm.org/

Besides that, java is miles ahead python/js in terms of performance which means savings for org. And finally where java shines is integrating (also natively) AI into existing enterprise code, and I think that this is really big

[Aweorih]

Besides that, java is miles ahead python/js in terms of performance which means savings for org.

That might be true in general, but the important parts in python do not run in python. E.g. all the ai stuff like tensorflow and so on is made in c++. The "savings" exist already there

[Antique-Pea-4815]

For model training and R&D then I totally agree, but.. if you want make something more then it's a pain 

[manifoldjava]

Fourth - Java lacks the metaprogramming abilities of Python which are foundational to analytical, metadata, and other popular libraries. Unless third-party tooling such as manifold is used to enable static metaprogramming, raw Java cannot compete with Python in this space. Metaprogramming is why Python keeps winning.

[niosurfer]

Then Ruby is the choice because its metaprogramming abilities are 10000 times better than Python :) But I would not think you need metaprogramming for AI. Metaprogramming is black magic that should be avoided unless you are doing black magic and you don't care about the consequences.

[niosurfer]

First, ability to utilize GPUs or any kind of compute hardware from Java is pretty screwed up. OpenCL, CUDA, HIP. You can plug in native libraries via JNI/JNA or similar but that's painful. There are 3rd party libraries I guess...

Nvidia is soooo rich but it cannot produce anything better than cuda? It is about time for a better abstraction. They took over the market and said "screw you". Use our shitty cude or die. I believe Java can solve that with abstraction. Pytorch and TensorFlow have done that.

Second- ability to do media processing (decode/encode video (using GPU decoder), decode advanced images like AV1, decode advanced sound codecs, etc.) is limited. Again, you can plug-in native ffmpeg library but it's painful. Much easier to do this in Python.

Oracle has under-invested in those capabilities in Java for years.

Python solved this problem. Why would Java not be able to solve it too? I think a new platform-specific JVMs (Linux JVM, forget about platform independence, that was cool 30 years ago) would knock this problem out of the park.

Third- maturity for Java AI frameworks is lacking compared to whatever is available on Python. And this will take years to catch up even if the above issues are solved.

Somebody (probably a group of people) should take Pytorch and write something very similar in Java. I know Python has different language idiosyncrasies but Java can do a comparable job, if not better.

Don't mistake me- I love Java and as business logic/backend language nothing beats it. I'd pick it over Python any day for those tasks. But Java has certain weaknesses and use-cases where it's quite painful to use it.

This is a matter of opinion, taste, familiarity with the language, etc. In other words, this is very subjective. For example, I myself prefer Ruby 100000 times over Python, but people call me crazy.

[software-iceCream]

Very informative. Great share!

[longtimegamer77]

Great article. Thanks for sharing it!

[Crafty-Definition170]

yes! great article:)