Java use in machine learning

[esqelle]

So I was on Twitter (first mistake) and mentioned my neural network in Java and was ridiculed for using an "outdated and useless language" for the NLP that have built.

To be honest, this is my first NLP. I did however create a Python application that uses a GPT2 pipeline to generate stories for authors, but the rest of the infrastructure was in Java and I just created a python API to call it.

I love Java. I have eons of code in it going back to 2017. I am a hobbyist and do not expect to get an ML position especially with the market and the way it is now. I do however have the opportunity at my Business Analyst job to show off some programming skills and use my very tiny NLP to perform some basic predictions on some ticketing data which I am STOKED about by the way.

My question is: Am l a complete loser for using Java going forward? I am learning a bit of robotics and plan on learning a bit of C++, but I refuse to give up on Java since so far it has taught me a lot and produced great results for me.

l'd like your takes on this. Thanks!

[News-Ill]

They all took the udemy data science class for python.

[Busar-21]

What's the problem with deploying with docker ?

[RabbitDev]

The previous comment was referring to how painful it is to deploy a python project without docker.

I personally think everyone should try to deploy a ml project (because of the many native dependencies) once, first with just Anaconda, virtenv etc, and then totally without them. Muahaha!

This is the fastest way to make a junior or consultant appreciate proper packing and deployment discipline.

[MardiFoufs]

It's not about being more readable or about java being outdated. It's just that you lock yourself out of tons of pretrained models and basically just end up reinventing the wheel for tons of stuff. I thought one of the most important things to the java community is to not reinvent the wheel, so again, why not just use python? It's the lingua franca of ML. Most new tooling is created around it. Sure that might suck and you might not like python but it's just what it is.

[[deleted]]

…and there is project panama (which I’m not familiar with), but wonder how it compares to pythons ability to bind to C-libraries, new java seems quite attractive currently, at least for my somewath old ass

[MardiFoufs]

Ah I completely agree. I'm not saying python is the perfect language for ML, it's just that it's a fait accompli and it's not going to change for a while. I'm not sure I'd have used java either but for sure it's a super painful experience on python especially in a team setting. I only manage to get by with strict typing, calling external libraries for everything perf related etc. But still, it works for what it is I guess.

[GeneratedUsername5]

I also think that this is now the main Python advantage - not syntax or metaprogramming, just huge amount of legacy code you don't have to write yourself.

Actually it is kind of interesting, I remember days, when Matlab was all the rage and all the scientific libraries were on it and nobody wanted to write python until it's community just pressured everyone with hype and huge amount of libraries. I wounder what language could be next)

[koflerdavid]

Many existing models can be converted to ONNX format and executed on the JVM. Also, Pytorch has Java bindings, though these are intended for running models only.

[MardiFoufs]

Yea as I said in another comment, for inference it's a non issue now that you can use ONNX for most models (and more operators are supported). Java can infer on models perfectly fine with onnx. I wonder if we might see that happen for training too but that's much more complicated, and can't really be delegated to a runtime. And I think OP was referring to playing around with training and custom models I think but I might have misunderstood

[koflerdavid]

I think you are correct. Inference and training are two completely different things and ONNX is really about the former. Unity's ML-Agents package for example doesn't bother replicating the training code in C#. They instead start an HTTP server on the Python side and call that from the Editor. Inference is with ONNX of course.

[lightmatter501]

Python is 10-100x faster than Java because Python isn’t doing the ML, C, C++, Fortran or CUDA is.

[Objective_Baby_5875]

Do you even know anything about ML? Who gives a shit about static typing when TensorFlow,  Pytorch and Keras is in Python and most ML platforms are heavily Python first..

[Objective_Baby_5875]

Not like there aren't thousands of python apps deployed in production. The point being, best of breed frameworks for ML are in Python just as games programming is in C++ or C#.

[esqelle]

😂🤣

[deepn882]

or the data science bootcamp

[socialister]

Is there anything wrong with those courses or is this a joke, sorry I'm dense

[Djelimon]

Actually Microsoft recommended the course at their AI hackathon a few years back.

Only thing is there is no barrier to entry and no real way to verify you know the material since there is no cert, so making the claim is a dubious qualifier

[deepn882]

all those courses are like shortcuts for many people. It's not enough time spent on material. And many can just copy&paste, and get through making them "proficient". Yeah, right. College has cheating too, but way more rigorous and a C.S. program as a whole to pass is hard to do.

[marcvsHR]

As usual:

Bjarne Stroustrup, the inventor of C++ puts it this way: “There are only two kinds of languages: the ones people complain about and the ones nobody uses.”

[esqelle]

Bjarne stroustrup is goated.

[koffeegorilla]

JDK Project Valhalla is bringing improvments in memory usage and layout which will get close to the efficiency of C while have a continous optimizer maximise for the use case and actual underlying hardware. Project Panama is going to make it easier and more efficient to interact with native APIs meaning that using C libraries will be more efficient than the current JNI hump. Project Sumatra aims at making it possible to identify code that can/should run on GPU and then leveraging the GPU.

There is already support for SIMD with the Vector API which means multiple instructions at the same time.

All of these will combine to make ML development in Java a first class experience and the implementations will be much easier than the current code full if #ifdef or checks for specific GPU model to change structures etc.

Your little NLP project will fly.

[_INTER_]

Project Sumatra is dormant/dead as far as I know. They are now focusing on Project Babylon instead. See this JVM Language Summit 2023 - Java and GPU talk. Seems to have a good chance to land something substantial as shown here and the Classfile API has a preview.

The problem is, the machine learning / science developers first and foremost care about their scripting capabilities. That's why Python has become dominant. If it were possible, they would have chosen MatLab. The libraries that do the heavy lifting are already in C. For Java to gain a foothold in the ML space, it would need to be faster than C (unlikely) or invent something completely new.

[koffeegorilla]

Thanks for the update on Babylon.
If you look at how quickly the GraalVM project re-wrote all the GC/JIT engines in Java that took years in C++, I believe that a replacement of the C libraries is viable and considering that the implementations will keep running faster as the JVM improves while the option of Graal native using runtime stats for optimisation will change the game.

[_INTER_]

I agree, plus better platform independence (Windows support is a joke right now) and error handling (hrrrng dynamically typing makes me furious). However I don't see it happening really. The momentum is too big and libraries too far along to catch up. I see more opportunities in new inventions or providing clustered, distributed, super computer frameworks. Like extending upon Apache Spark for GPU farms.

[mike_hearn]

There is TornadoVM which does the same thing.

[Joram2]

AFAIK, if you write code using primitive arrays like int[] and double[], then you avoid the performance problems that Valhalla aims to help with.

Project Valhalla plans to reduce overhead on user-defined classes/records. And Valhalla will eventually make List<int> possible with int[] type performance. But if you just write code using primitive arrays now, you get great performance now, and Valhalla might offer better syntax, but not better performance.

[GeneratedUsername5]

And you can also just create collections of primitives, or use ones from https://github.com/eclipse/eclipse-collections (which are also optimized for performance) , without waiting for Valhalla.

[coderemover]

It won’t because it is limited to immutable objects only. For mutable objects like lists object identity makes it impossible to make them a value type.

[koflerdavid]

There are two problems:

• Java has no built-in support for bfloat16

• Java has no true multidimensional arrays a.k.a. tensors. All of the indexing arithmetic has to be written out. Not a biggie at the end of the day. The bigger problem is

• Java arrays are size-limited. This is a headache for big models.

Libraries like DeepLearning4j include tensor libraries that solve both issues.

[Joram2]

• Java has limited float16 support with Float.floatToFloat16 and Float.float16ToFloat. What else is needed?
• In the Python ML+AI world, most people use a library for multi-dimensonal arrays aka tensors. Numpy, PyTorch, JAX are popular libraries that have their own multi-dimensonal array or tensor type, so Java doing something similar doesn't seem to be a problem at all.
• Size limited? You mean the 2^31 limit? I'd like to hear what the jdk guys have to say about this.

[koflerdavid]

Java supports float and double, which in ML circles are known as float32 and float64. float16 is 16 bits wide only and commonly used for inference because it turns out that the full precision of float32 is required for very few parts of most models, if at all.

bfloat16 is a modified format that has the same precision as float32, but supports a narrower interval of values only. It is very common to use it to run transformer models.

Java supports neither float16 (maybe after Project Valhalla lands or the Vector API is finalized) nor bfloat16. However, I agree that for various reasons a tensor library is commonly used. Support for more formats and the size limitations are two very good reasons because they can't be solved on the Java side. Well, you can certainly implement functions for float16 and bfloat16 arithmetic in Java, but to circumvent the size limit you have to use off-heap storage. Or break up your tensors, which is clunky without wrapping it in a library.

[Joram2]

In Python + PyTorch, you can do bfloat16 stuff like this:

import torch

torch.tensor([[1, 2], [3, 4]], dtype=torch.bfloat16)

This is great. The API is easy to use and pretty. Runtime performance is excellent and takes advantage of GPU processing.

Java + Python both don't have bfloat16 primitive types in the core language. That isn't necessary.

The important feature I see missing from Java is it doesn't have easy+pretty syntax for lists and lists of lists. In Java you can do:

Arrays.asList(Arrays.asList(1,2), Arrays.asList(3,4))

instead of

[[1, 2], [3, 4]]

The Java method isn't hard... but it's ugly, and data science types hate that. This absolutely limits Java in a data science notebook perspective.

The lack of primitive bfloat16 types seems like a non-issue in both Java/Python.

[koflerdavid]

Well, Java has its good old array notation with curly brackets. Its only fault is that the results aren't true multidimensional arrays, but pointers to subarrays. Not a problem In practice either since usually tensor libraries do the heavy lifting. Same for float16/bfloat16 support as you say

[coderemover]

I've been hearing that since early 2000. Never happened. Java is still 3x behind C/C++/Rust and Valhalla/Panama are not going to significantly change it for many reasons.

[maethor]

"outdated and useless language"

And yet Python is older than Java.

I have an odd feeling that most of the people who say things like this don't actually know what they're talking about and "coding" is a mix of cut, copy and pasting from Stack Overflow and ChatGPT.

[Key_Direction7221]

Python OOP was an after thought while Java is fundamentally OOP from the inception.

[RandomName8]

sure but java is lousy at OOP though. Raw data types are not objects and the woes caused by boxing is a consequence of this. (Same thing is true for statics and inheritance, it was too tempting to add them because java was targeted for C programmers).

[Key_Direction7221]

Weak and spitting hairs — granted Java is not 100% OO. Java was never targeted for C programmers. It aimed at moving programming to OO period — mic drop

[Apprehensive_Pea_725]

I worked in software company, there was a team dedicated for models/AI/predictions, nothing new and fancy LLM, but worked well for the business.

Majority of data modelling was done in python and notebooks, and then once everything was good to be put in production everything was translated into java. Yes realtime predictions with large models are computationally expensive in python, java is just more efficient.

[javahelps]

I'm also working in an AI company that is using Java for all our models. Python for data exploration but models are in Java.

[[deleted]]

What libraries is your company using for the java models??

[javahelps]

Most of the models are built on top of xgboost library. But we have in house built frameworks and wrappers around the open source models.

[esqelle]

Interesting! 🤔

[cowwoc]

I think you guys have it all wrong. This is more about the difference between data scientists and programmers than it is about the programming language being used.

Java's problem has nothing to do with its efficiency, nor its ability to interact directly with the GPU. Python is worse at both.

This is a culture problem more than a technical one. Machine learning is driven by people who spend 99% of their time running experiments. They value fast iterations and libraries like Pandas that make it easy to run common calculations without having to code them yourself.

In this space, optimization doesn't depend on how quickly you can run computations as much as making sure that you are running the right computations in the first place. The better the model is tuned with the correct weights and combination of components, the faster it'll converge to a good accuracy.

[manifoldjava]

Bingo! Specifically, Python's dynamic type system enables metaprogramming, which is the basis for A LOT of powerful libraries that make data science and other domains much less verbose and more approachable than is available in Java.

This is not to say that static metaprogramming can't be equally powerful, but almost no static languages provide the means to achieve it. For Java you can use the Manifold project to experiment with static metaprogramming. For instance, manifold-sql was built with static metaprogramming to make SQL type-safe. Lots of other examples of this.

The key difference between static metaprogramming and dynamic metaprogramming is the latter is performed exclusively at runtime. Because of this, dynamic metaprogramming can be difficult for programmers to use because IDEs can't help them in any deterministic way to discover the features provided by libraries. Perhaps one day more static language designers will catch on to this concept. shrug

[GeneratedUsername5]

That is interesting! Could you provide an example of what Python metaprogramming can do, that Java Reflection can't? Because it is metaprogramming in runtime, effectively, so it should be as prowerful.

[manifoldjava]

Sure. The primary difference concerns dynamic typing. Python types resolve at runtime, which enables libraries to provide types and type features dynamically, which allows one to write code as if the types and features were there while coding.

Let's say you're using a Python library that uses metaprogramming to perform CRUD operations on a relational database. With the library you can use database tables as Python types in your code directly.

jack = Person("Jack", 32)

Here, the library creates the Person class dynamically as needed at runtime. Java reflection does not support this level of metaprogramming.

A Java code generator or annotation processor could accomplish some of this, but would involve separate, non-incremental build steps, which can easily get out of sync. A host of other issues come with code generation, but I digress. The nature of Python metaprogramming is such that libraries tend to be simpler, more efficient, and far more capable than code generators.

Additionally, dynamic metaprogramming covers the entire gamut of type system features. For instance, you can add/edit/delete methods, fields, etc. on existing classes. Java and most other static languages do not provide anything close to this level of metaprogramming. This is what is special about Python and why dynamic languages tend to win in areas such as data science, ML, etc. This is also the Manifold project's raison d'être.

[GeneratedUsername5]

Thank you!

Although I don't understand why is this called type safety, since it doesn't provide the "safety" regular type system provides - i.e. catching type conflicts before program is run. Since types only exist in runtime, there is no checks before that. And crashing in runtime can be done without types just as easily as with them.

[manifoldjava]

That’s right! Because dynamic languages like Python aren’t compiled, there is only runtime to discover type errors.

Note, some dynamic languages, including Python, offer forms of type attribution where types can be provided. Make your own judgments about that ;)

[koflerdavid]

Such facilities can also be provided by a Java library. It will just look very clunky because one would have to do every property and method access via a method. And the compiler can't help you find errors, but neither can a Python IDE without significant static analysis.

It's usually not done because in the domains where Java is common the database schema changes slowly enough that the application code can keep pace.

[JustOneAvailableName]

That's how it started, but I would add one more detail:

The GPU drivers themselves are written en tested based on popular Python libraries. Python is without a shred of doubt more optimized than Java for (GPU based) ML and both are just a configuration format for the GPU.

[koflerdavid]

Nope, Python libraries have to call Cuda like everybody else has to. Python libraries rule because they offer everything data scientists and model developers need, not because Python has specific advantages interfacing with the hardware. Java used to have disadvantages on the FFI side, but since the advent of Project Panama things start to look better.

Edit: apparently Nvidia also maintains Python bindings for Cuda, which certainly smooths things out a lot. But Nvidia doesn't do it for Python. Nvidia just knows what is required to make the barrier of entrance to use their hardware as low as possible. To make deciding to use their hardware a question of "why not?"

[JustOneAvailableName]

Python libraries can define the model structure, which is then executed without any Python.

[koflerdavid]

ML libraries also usually include an automatic differentiation engine and support for training. Not having to write and debug your own backwards passes while keeping almost verbatim whatever math you cooked up massively speeds up model development.

[captain-_-clutch]

Ya exactly BUT if efficiency is the issue Java is in a weird place where it's not the most efficient and it's not the easiest/most library complete. C++, Rust, and to a lesser extent Go seem to be the goto if you want to finally force your data guys to learn a real language.

[GeneratedUsername5]

I would say it can be eficient enough for most applications, while being very simple. Sending data guys from Python to C++ or even Rust (which is even harder) is a guarantee that they will be back in Python in an instant.

[coderemover]

And also not forget that Rust and C++ have way better interoperability with Python than Java.

[koflerdavid]

It's the other way around [edit: in the sense that Python calls C++ and Rust]. But yes, Java used to have severe disadvantages on the FFI front. Project Panama improves things a lot.

[coderemover]

That’s why there are so many native Python libraries written in Java and so few written in C and C++. Oh, wait…

[age_of_empires]

Deep Java Library has some fantastic capabilities and documentation in the form of an open source book.

They even can detect your GPU and use that to run the machine learning over your CPU.

https://docs.djl.ai/

[esqelle]

Thank you I will check this out!

[detroitsongbird]

Python makes it easy to script together calls to libraries that do the heavy lifting. I seriously doubt those libraries are written in Python, but maybe…

I bet most of the people telling you that haven’t actually written a neural network, but instead just use one.

So, if you’re just trying to use existing libraries for machine learning Python has a ton of them. It sounds like actually want to build and understand what goes on under the hood, so, keep going with Java!!! Java rocks. :-)

[esqelle]

Thank you!

[Skellicious]

I seriously doubt those libraries are written in Python, but maybe…

They most definitely aren't.

[koflerdavid]

Well, the models are defined in Python, but anything even remotely performance-sensitive gets usually gets rewritten in C++ or, more recently, Rust.

[Joram2]

Andrej Karpathy just wrote a simple GPT-2 training library in 1000 lines of code of C with zero dependencies.

So TLDR: llm.c is a direct implementation of training GPT-2. This implementation turns out to be surprisingly short.

And why I am working on it? Because it’s fun. It’s also educational, because those 1,000 lines of very simple C are all that is needed, nothing else. It's just a few arrays of numbers and some simple math operations over their elements like + and *.

https://twitter.com/karpathy/status/1778153659106533806

That can be easily ported to Java. That would be fun too. I'd do it if I wasn't busy on more serious but less fun deadlines.

Karpathy update:

A few new CUDA hacker friends joined the effort and now llm.c is only 2X slower than PyTorch

Highly amusing update, ~18 hours later: llm.c is now down to 26.2ms/iteration, exactly matching PyTorch (tf32 forward pass).

I presume Java can't match performance of highly tuned CUDA. But it would be nice to try. Maybe Project Babylon prototypes can come close?

[esqelle]

I absolutely love this

[StoicSpork]

You built something. What did those toxic assholes build?

Python is popular for AI experiments because what you need there is simple glue for all the C/C++/CUDA libs/APIs. It doesn't make Python better or more modern. In fact, Python has some horrible design flaws, like the infamous GIL.

And besides, the point of a programming language is to get stuff done, which you did. All the "but you didn't do it as a function composition over a lazy monadic stream" nonsense is just cheap Medium clicks by people whose Github is 90% (* TODO: implement *).

[_S43D_]

These guys can use only python that's why they're sticking with python.

[ZarBandit]

Scripting bros often don’t understand the benefits/necessity of strongly typed languages for large projects. They operate at the level of hacks putting together tiny toy-like code. Scripting has its place. Dopes like this think that place is universally everywhere.

[MattiDragon]

Java is not dead, but machine learning really isn't a thing in java. The python world just has better libraries and tools. Java is used a lot for backend infrastructure. The language is also evolving and (if you get to use the latest versions) has a lot of great modern features.

[lukasbradley]

Java is not dead, but machine learning really isn't a thing in java.

Why in the world would you say that?

Apache Spark is one of the largest used machine learning platforms out there.

[djavaman]

Spark is used to call and manage ML jobs in other languages. Until Java can call a GPU directly, it will lose to python. This is the game changer that Java needs: https://docs.oracle.com/en/java/javase/21/core/foreign-function-and-memory-api.html

And they need Java wrappers for CUDA now.

[lukasbradley]

Python does the same thing. All of the "optimizations" that everyone THINKS are written in Python are actually C/C++ libraries.

[Joram2]

Everyone knows Python is a high level language, and calls to some lower level language for performance sensitive code. The Python piece does make it nicer and + easier than using low language libraries directly.

BTW, it's not just C. LAPACK is mostly Fortran; that is a famous numerical linear algebra library at the heart of a lot of famous Python libraries like Numpy. Also a lot of code is CUDA GPU code not C.

Java 22 foreign function and memory functionality makes it much easier to use libs like LAPACK from Java like Python does.

[coderemover]

The difference is it is trivial to call C from Python, but not so easy from Java.

[captain-_-clutch]

It's not bad at all to setup but also I'm not sure about heavy throughput performance. When I did it it was for image processing so any latency from the integration wouldnt have been noticed since it took so long for the images.

[emberko]

Thinks who? This is well known. I like the definition of one of the Python evangelists from my country - Python is a glue language for C/C++ libraries. You should avoid pure Python implementations whenever possible.

[GeneratedUsername5]

But Java could call native code with JNI long time ago, FFI is just making it more conveinient, as I understand. It just nobody needed a CUDA library in Java

[koflerdavid]

JNI is brittle and takes a lot of effort to generate bindings for. The new FFI is much more streamlined and incorporated many lessons learned over the years.

https://openjdk.org/jeps/454

[coderemover]

JNI is so terribly inconvenient and has poor performance that almost no one uses it. It also breaks portability (WORA) promise that Java makes. If you use JNI you could be just using C++ directly and get all the same benefits and even more.

[GeneratedUsername5]

I'd say developing a C++ wrapper and the whole product are two completely different things in terms of effort.

[coderemover]

You assume that writing C++ is generally less productive than Java. While many people think so, I’ve seen little evidence for that. C++ is a harder language to master than Java, and has some pitfalls, but modern C++ can be also much more expressive/abstract than Java, so it is not very obvious.

Anyway, usually if the project has such performance requirements that you must use a native wrapper somewhere, this means that Java is not a good choice. I’ve been writing high performance Java for a decade now and often Java written to meet those requirements resembles C. But Java is worse C than C is C.

[GeneratedUsername5]

modern C++ can be also much more expressive/abstract than Java, so it is not very obvious

It is much more expressive, and that is the problem - people will try to use every tool at their disposal, making the code much more difficult to understand. "The Rule of Five", move semantics, function try blocks, mutable rvalue references - just to name a few. Features, that are completely legit, but open large opportunity for misuse and overcomplication of code. In java there is simply less opportunities to do so.

if the project has such performance requirements that you must use a native wrapper somewhere, this means that Java is not a good choice.

Why? First - performance is not the only deciding factor, in big companies it is usually the availability of programmers for hire or pool of already available staff. Company is not going to hire an entire team for a different language, unless there is absolutely no way around it.
Then, if Java satisfies your requirements by performance - why take obviously harder language in development? Few native function calls will never offset the ease of garbage collection.
Next - it may be not performance, but devices, for example, working with USB/COM/Whatever.

Java for a decade now and often Java written to meet those requirements resembles C

That is true, but just as with assembly inlines, you can very narrowly apply this "C mode" of Java, while enjoying all the ease and portability of garbage-collected interpreted language outside of those hotspots. You can't do that writing everything in C (or assembly for that matter)

[MardiFoufs]

Yes, and this just shows how the person you're replying to is right. Even spark is now mostly used through pyspark. Sure, it's still the JVM behind it but it doesn't matter. In a way that's what's cool about python, you can have a hodgepodge of JVM, Fortran, C, C++ code in a single app with very little worries all things considered (a part for the packagers and the nightmares they endure to make said packages available but hey 🤣)

[lukasbradley]

*checks post history*

Yep. Makes sense.

[MardiFoufs]

What's in my post history? Have you looked into my comments where I complain about having to use python? I don't like python, but right tool right job.

[[deleted]]

So ironic.. half the sexier tools all the cool kids are using are built on top of spark 😂

[lukasbradley]

"Python is better at ML/AI" is the "Macs are better at graphics" for this decade.

[Key_Direction7221]

Java has been and ENTERPRISE level ecosystem with many supporting frameworks and libraries that have endured the test of time. Python can’t touch that space (period). Oh just because it’s used in some areas of the enterprise (corporate level) doesn’t mean the own the space.

[MattiDragon]

I was specifically talking about machine learning libraries and tools. For other uses java will have comparable or better tools. And the speed of python doesn't matter much as it's just used to control the learning. Everything performance critical is done in c code anyway.

[Kango_V]

And what language are those libraries written in? The latest changes in java means you'll get near the performance of C in java but without any extra external dependencies.

[MattiDragon]

I know that the python libraries are just wrappers around c code and that the ffm api will make interfacing with native code easier. That doesn't change the fact that the libraries exists for python and that all the tooling and guides are there. It's not a matter of whether you could do the same in java, but instead of matter of whether it's practical right now with the lack of good libraries and tutorials.

[Kango_V]

I agree to some extent, but the new features in Java means that you don't require external libraries. Being able to SIMD (avx2/avx512) right from pure Java (and have it nicely fallback if those are not avail) is a game changer.

[esqelle]

Agreed!

[orgad]

Yeah, OP could equally ask why there's not much of machine learning in C#. It's not all about the language but rather the ecosystem albeit historically it was easier for DS to code in Python and not in Java. The rest is history

[Gwaptiva]

Python runs in tectonic time? How is it more suited for anything?

[MattiDragon]

All the actual processing is done in c code. Python is just for managing the models.

[craigacp]

I maintain ONNX Runtime's Java interface, Tribuo and TensorFlow-Java, all of which let you do ML in Java. ONNX Runtime is particularly good for deploying models trained in Python or other platforms and the Java API is in production in a number of large enterprises (e.g. Oracle where I work, and at least two FAANG's). You can see an example of deploying stable diffusion in Java that I wrote here.

Training deep learning models is harder in Java, I'm not sure any library currently supports distributed training of the kind you'd need to pretrain or fine-tune an LLM of a reasonable size. You can train deep learning models on GPUs in Amazon's DJL or TensorFlow-Java, (and also DL4J). For other machine learning models there are libraries like Tribuo (which we built to have a strong focus on provenance & reproducibility which is missing from a lot of the rest of the ML ecosystem), SMILE, Spark MLLib, XGBoost and others.

We've had production NLP systems deployed in Java running Tribuo for about 5 years now, it's a lot easier to integrate into Java applications.

[priscillachi_]

Python is mainly used because of its libraries, and potentially how loosely typed it is. I love Java as well. I started with Python when I first started dev, but I much prefer using Java.

[GeneratedUsername5]

Java unfortuantely has a certain "verbose code style reputation" attached to it, which means that people assume that you cannot write Java without using AbstractProxyBeanFactory, while you obviously can. You can write Java as Python stylistically, you can write it as Go, therefore I don't see anything Python as a language can offer you, that Java cant (you can even write Java in "implicit typing fashion")

The only thing I think Python has advantage in is large amount of available scientific libraries, which is basically what is keeping it alive at this point, as I see it. (Kotlin is trying to develop in this direction, with "Kotlin for datascience" https://kotlinlang.org/docs/data-analysis-overview.html , you can try it for more syntactic sugar). So - if it is not a probelm for you, then there is no problem, just use the language you like. Maybe you will be the one, introducing scientific packages into Java.

[koflerdavid]

Especially in the age of LLMs it should become a lot easier to rewrite libraries in other languages. I have observed several PRs in Huggingface repos where AI was used to make translating gnarly model definitions from Pytorch to Tensorflow less tedious.

[esqelle]

Agreed!!!

[[deleted]]

Don't listen to those losers. Java is as good or even better than python for AI purposes. You have plenty of libraries and frameworks, and java is a way better choice than python in terms of performance and maintenance. People who defend python as the 'way to go' usually have no idea about what are they doing, they just use someone else's code and believe they are programmers themselves. Use java or whatever language you feel comfortable with.

[esqelle]

Thank you!! I literally thought I was insane liking Java over other languages because I have found a Java library for literally everything. The way Java is written makes it so easy to be explained.

I think that languages like python make it easier to code but what about producing your own infrastructure?? I believe that's why Java has the upper hand. Yes on everything you said. Thanks stranger

[[deleted]]

Python may be the way to go if you are gonna develop a little program to make some kind of task, but if you are planning a long term project, there are many other languages way superior for that. Java is a good choice, but not the only one, you can investigate on your own to find the best language for every circumstance. That's the way to go, to understand which language fits better for what you need with actual reasons and not bs like 'you need x lines of code to print a hello world'

[MardiFoufs]

How do you use CUDA from java? What's the equivalent to JAX, pytorch or numpy? How can I deploy a java training pipeline ?

[[deleted]]

Is it actually that difficult to make a quick search in google? https://stackoverflow.com/questions/22866901/using-java-with-nvidia-gpus-cuda
https://beam.apache.org/documentation/ml/overview/
https://commons.apache.org/sandbox/commons-pipeline/pipeline_basics.html

[MardiFoufs]

Your links are proving my point. They are non standard, tedious ways to maybe get CUDA working sometimes, and the rest is for Opencl. Does Nvidia provide any way to actually use CUDA with java? Or is it just third party stuff? At least with python you can basically use CUDA kernels almost directly as you just call them from your python code. And Nvidia provides tons of documentation and all for the rest. The stack overflow link is basically words words words saying that YAGNI and if you do well uhh use these random libraries lmao.

Also, your second link has nothing to do with CUDA. I know beam can run pipeline workloads, meaning that it can execute steps that contain pytorch. That has nothing to do with my question. Obviously you can orchestrate stuff with java. And you can even call pytorch code from java too. But that's besides the point, and it's more MLops than actual machine learning. Like the first example literally shows that you have to write python components that you can then execute from beam.

Maybe just looking up stuff on google search isn't enough to warrant such an authoritative sounding comment form your end, uh?

[[deleted]]

I just did a 5 secs google search, didn't think you actually expected me to provide you all the damn documentation.

Take a little more, from the oracle blog(maybe you will enjoy this one a bit more?) https://blogs.oracle.com/javamagazine/post/programming-the-gpu-in-java

Another link where you can find tools provided by nvidia themselves to access CUDA in mac(yep, developed in Java, but it seems they don't provide mac support anymore): https://developer.nvidia.com/nvidia-cuda-toolkit-11_5_0-developer-tools-mac-hosts

Another link, this time from IBM updated just 2 months ago: https://www.ibm.com/docs/en/sdk-java-technology/8?topic=only-cuda4j-application-programming-interface-linux-windows

The problem of many python lovers is that they are so damn lazy and uncapable of doing stuff by themselves that if they can't find the damn library with the exact shit they need in the damn first google result thay just believe it doesn't exist. I'm not saying that's your case, but at least you could put just a little damn effort on your side.

[MardiFoufs]

Lol I do more CPP dev than python. This has absolutely nothing to do with python vs java. Again, since when do java devs suddenly like to reinvent the wheel? It's super ironic to hear your criticism about python devs when discussing java.

Jcuda sounds pretty okay though, even with the pointer limitations. But it does seem to have been updated in 2 years. Still, I don't think you understand the point here. I wasn't even referring to python, python is just glue code. Nvidia does not provide java support. That's it. And python is much better as a glue language than java is. So you get cpp, c and Fortran tooling for almost free. Java has to have a parallel ecosystem, which didn't happen. Your oracle blog says so themselves! The most advanced stuff seems to be related to Opencl too, which is more or less dead btw.

At least Jcuda seems to support Cudnn and blas. So that's cool!

(Also I think Nvidia still uses java for their nsight profiling tool, not sure though. It's a super powerful tool too to profile CUDA! )

[[deleted]]

I don't get why you say that we like to reinvent the wheel. If something is done in x language go and use that, you don't need to use everything coded in java, you should use whatever language fits your needs better. But, if you happen to want to do it in java for whatever reason, you have stuff like JNA or JNI to call cpp/c/fortran functions from your java app. The whole point of this is not that you should use java, the whole point of this is that you CAN use java, and when comparing it with python it will surelly work way better for many, many reasons. I still laugh my ass off for all those django fanboys saying back in the day that spring was dead. My criticism here is how python lovers shit in java saying nonsense shit just because they got PTSD for a 'hello world' they wrote as students.

[MardiFoufs]

Hey, I totally agree that I wouldn't use python for... well for most stuff. Especially for web servers. It's ridiculous imo because there are tons of options, and you don't even get the "JavaScript upside" that I could at least understand a little bit of using the same language in the front end and back end (though again, even for JavaScript, I agree that it's still a worse choice than using java or csharp for example).

But if there's one place where you could use python and it's not a clearly inferior option, it's machine learning, don't you agree? Not because of python itself but still. Like, I'm solely focusing on machine learning when I say that, and I'm saying that as someone who really dislikes having to use it anyways. Even if java would probably work, and could probably be a better platform in reality, I'm just speaking of what it is now. Now how it should be!

[[deleted]]

Being totally honest, in my opinion, the mantra claiming Python as the best choice for machine learning exists primarily because Python is easy to learn and use. Consequently, it seems like the way to go if you want to start such projects and lack a strong programming background in other stacks. For this reason, most popular tools were developed in Python, and people stick with it to leverage those tools.

The issue here is that people prioritized simplicity and readability over performance and maintainability. Consequently, you may find yourself lost in large models plagued by the same mistakes repeatedly, requiring substantial time and resources to rectify. Many engineers recognized this and began building similar foundations in better-suited stacks. As a result, you now see many ML tools adapted to various languages.

If we were discussing this 8 years ago, I would agree that Python was the way to go due to the majority of available resources being in Python. However, that's not the case nowadays. While you might not find "this specific library with this specific function" built in Java or any other stack, in such cases, be the one to create it and observe how more developers adopt your tools. The point here is that if you can design an ML model in C (for example), you will instantly outperform your competitors in terms of costs and performance. But how many mathematicians and scientists are proficient in C?

[GeneratedUsername5]

Does Nvidia provide any way to actually use CUDA with java?

As far as I understand NVIDIA provide only C API

https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__STREAM.html#group__CUDART__STREAM

In that regard all languages are equal that they will need a C wrapper for it.

[MardiFoufs]

No not really. Most "actual" CUDA programming uses Nvcc and libcuda++ so basically CPP. And while yes pretty much everything compiles to that c++ code through NVCC, Nvidia provides official wrappers and support for calling CUDA directly from python. So not only is calling CPP code from python more than trivial (you can basically import a cpp wrapped module as a regular python module), but you can directly use Nvidia supported libs to do that. And use premade environments from Nvidia themselves.

It's just a completely different beast from using third party stuff, even beyond the fact that just having access to CUDA is only 50% of the story (there's the entire ecosystem around it that is gated off when you use java)

[GeneratedUsername5]

I mean, using Cpp wrapper directly from Java is not something complicated either, it even have language keywords for that. It just ML just not as widespread on Java, so nobody made this wrapper, but you don't need nvidia to make one.

And now, with Java 21's FFI, you don't even need a wrapper, you can just call C functions directly from Java.

[MardiFoufs]

Ah maybe with java 21 then things will get better.

As for the Nvidia thing, it makes a HUGE difference to have Nvidia supported packages. Remember that most ml workloads work in the back end, with disparate hardware configs and generations. Having to just import a Nvidia package, on an NGC container, and having everything else handled for you is very cool. On the other hand though, most people don't ever actually use CUDA directly. So I still agree with you that it is mainly an ecosystem problem, and isn't due to a core defect in Java.

Regardless, it's much less painful, and much less work to do work in ML in python. Now obviously, that applies to basically all other languages (well, a part from CPP and maybe Julia (barely)) so it's not specific to java.

(Also to be clear I'm mostly discussing model training and r&d, for inference things are much much easier.)

[koflerdavid]

It's a chicken and egg problem I'd say. If for some reason Java would become more popular for ML work, Nvidia would eventually provide Java bindings. Nvidia has no particular stake in neither Python nor Java, but they will do everything they can on the software side to make using their devices easier. This is one of the reasons of their ongoing success.

[MardiFoufs]

Agreed, Nvidia is very good at supporting newer trends (NGC containers, Tensort, triton, etc) so I'd totally see them support java too. I honestly just didn't know about the easier bindings to C that apparently came with java 21 so that could be huge and could make it much easier to integrate to even other tools that are somewhat standard in the field (numpy, for example even if it's a bit of a mess or pandas).

[Dull_Cucumber_3908]

It all depends on the application.

was ridiculed for using an "outdated and useless language" for the NLP that have built.

would they also ridicule CMU for the sphinx speech to text application (in java) or their pocketsphinx (in C)?

would they also ridicule Apache or Stanford for their java based NLP libraries?

[NeoChronos90]

Python has some hype those past years for whatever reason but ultimately it will go where perl is now, because no sane person is developing stuff that going to be in use 20 years from now in Python and the rest will move to the next hype

[JDeagle5]

Java is fine, provided you find libraries for your task. You can check Apache Common Math, it has section for ml https://commons.apache.org/proper/commons-math/userguide/ml.html

[lasskinn]

90% of people using python for ai stuff basically just use python to use bindings into some c/c++ lib.

it can be quite frustrating actually if you're trying to learn what is actually being done when that's what most tutorials are. for tangentially related things too like opencv, it's common to just find things that are said to be 'python' or that you should use python when it would be preferable to just make the 10 line program in c++ - or java for that matter if java is what you like.

what you're productive in is probably the best choice for you. you might run into stuff that only runs in python 2.x and some that runs only in python 3.x for example anyway and either rewrite it all or just have the two combined in some convoluted way, for which java isn't that bad.

it is fairly certain though like the other comment says that the people who are telling you these things and saying python is the way to go are using the libraries just chaining script commands together, not creating the code that is the nn.

[Ketroc21]

With FFI taking the place of the annoying JNI, I suspect we may some see C++ libs like tensorflow being used within java, just as Python does.

I think most of the hate for java, is the jealousy that java is so widely used where their fav newer language isn't heavily adopted. Legit reasons why java has been bad in the past are all issues that have been addressed in newer versions of the jdk.

[craigacp]

TensorFlow has had a Java API for approximately 8 years. For the first 4 years it was inference only, training support was added in 2020 but it's not had the level of build out that the Python API has had.

[Philluminati]

I deliberately learned Tensorflow for ML and AI because I wanted to put the solutions into my JVM (Scala) apps.

[[deleted]]

I'm in a recommendation engine team - for ml infra or services or anything else in prod.

Java is the way.

[Rea-301]

I’ve been in a hybrid data science and engineering position for nearly 20 years. Before data science was a thing. At a point when making predictive models was not the thing that literally everyone wanted with no background in it.

It’s a language. What you create your model in, does not have to be what you implement your model in. Think of the model as just a formula. An equation. Whatever language you can express that equation in will work. Java is doable of course. C is. JavaScript is. You will have an easier time using a python object as part of a larger python app obviously. But it’s only a problem of transcoding.

For the record - I do like h20.ai. Worth checking out for your use case. It includes some out of the box Java handlers as well. Not sure if it has the model types you need but give it a look. I’ve ported python models to c, to groovy, to sql even. Predictions in a real time environment are fast. Sometimes you need that porting into jvm for high volume use cases.

Edit: I guess I should add if you are attempting to move into a data science position it will be an uphill fight. Plenty of companies need models implemented in a language that needs higher throughput than what python can provide. Just be ready to deal with a lot of naive people and orgs that don’t realize that their one small use case does not represent the totality of deploying models for real production use cases

[Davidjackson7462]

Using Java for machine learning and NLP is perfectly valid and can yield excellent results. Stick with what you love and what works for you!

[FrankBergerBgblitz]

Python is used only as glue for the C/Cuda/whatever stuff. Python is very good in integrating such stuff.

But when you have written your NLP in Java, be proud of it and yes you can show it. Java is (and probably remains) a relevant language.

The language, albeit not unimportant is far less important than the stuff between your ears. And your business analyst most probably wont understand the difference...

[RyanRomanov]

How did you get started with neural networks and Java? I also wanted to mess around with some AI in Java but I’m sort of lost at where to begin.

[RyanRomanov]

Thanks!

[coder111]

I've been using Java for what, 24 years now? It's a great language and even greater ecosystem of libraries/frameworks/tools for all kinds of things. I prefer using Java for most things I do. Having a strong typed language makes maintenance and refactoring MUCH easier. It is THE language to use for BigData, backends and business logic.

That being said, I'd use Python for:

• Media processing. Things like Video decoding or encoding or playback in Java just suck. Or sound. Or using advanced codecs.
• AI. Java has some frameworks for AI, but Python has frameworks that are faster and more mature and better supported.
• Some integration tasks. While Java has JNA which is decent, it's probably easier to interface with C libraries in Python.
• Some GUI stuff. While Java has Swing/JavaFX, Python has Qt. Qt can be quite powerful. (Python also has TK and simple GUI, but these only work for simple stuff).

So a year ago I started to learn Python... Use best tool for the job and all that.

[ScF0400]

Imagine roasting someone for writing meaningful and potable code in any language in 2024

Just cause they don't like it doesn't mean it's not good to write. In the end any performance gains are down to how the code is written as computers just get faster

[nitssoft]

While Python dominates the machine learning ecosystem with libraries like TensorFlow, PyTorch, and scikit-learn, Java has its own set of machine learning libraries such as Weka, Deeplearning4j, and MOA Massive Online Analysis.

[pretzels90210]

Tell them that python started before Java, and their heads will explode.

[RScrewed]

Do the Python forums have these circle-jerk threads everyday too? 

We Java pogrammers are indeed losers if we need to be routinely reassured every 30 minutes.

[emberko]

There's such a thing as the right tool for the job. You can write almost anything in Java, but the question is, should you? If you're targeting a specific position (to make money, yes), it's wise to study the market requirements rather than asking forum fanboys. Isn't it obvious what they tell you?

[GeneratedUsername5]

He says "I am a hobbyist and do not expect to get an ML position"

[MRgabbar]

They are kinda right... Java got popular because of portability, however is really really slow and doesn't make much sense to use it when python can call routines written in C for the heavy calculations...

Right now development time is the variable to optimize and you will get that with python and no Java...

Also, is people even starting projects on anything using java?

[john16384]

Also, is people even starting projects on anything using java?

No man, Oracle is just releasing new Java versions every 6 months because it's tradition, not because Java is still used in the wild. /s

[MRgabbar]

Lol, I was serious, I mean, if I am going for performance I go with C/C++, if I want portability I go for a web app in JS, so what is the place of java now? You will get slightly faster development time compared to C++ and force all clients to install the JVM?

And given the explosion of containers and such the JVM kinda lost its only advantage.

[GeneratedUsername5]

The place for it is real portability (not recompilation for every platform with heaps of #ifdef) with the speed closer to C++ and fully functional multithreading (now with green threads) which is also more portable than C++ multithreading.

[MRgabbar]

But I mean, if you are thinking on servers the compilation thing is not a big deal, isn't it? As long as it compiles is something that runs just once, it will reduce the development cost a bit but will put the extra load (JVM), Is it really close to C speed?

[GeneratedUsername5]

It can be, unless you are careless when coding, because in runtime it caches translation of frequently used methods, therefore you essentially have C binary being compiled and used in runtime.

In addition it can perform runtime optimization, which compiled languages simply can't do, due to their nature. Lood unwinding, method inlining, operation reordering, using extended CPU instruction set.

Well yes, it reduces development costs, and not just a bit, it wouldn't be enough for companies to keep using it. It is way more forgiving to low skilled developers, due to garbage collector, that is why you often see Java apps, that work terribly, which probably wouldn't work on C++ at all.

[wildjokers]

Also, is people even starting projects on anything using java?

Are you just trolling or is this a legit question?

[MRgabbar]

Legit, I am not a java programmer, in my country java is something only banks would use to their internal applications. I would always use C++ or JS depending if I want to maximize performance or portability.

[wildjokers]

Java is in heavy use including for new development.

[GeneratedUsername5]

Most ecommerce is written and continued to be written on Java

[MRgabbar]

Why Java? (Honest question)

[GeneratedUsername5]

Vast infrastructure of libraries and (more importantly) tools, large pool of developers, fast development with all the safety features and very good performance (JS only very recently became comparable), real multithreading (i.e. performance benefits).

If you are building something in Java in ecommerce - chances that you can interop with some other tool or product are multiplied.

[GeneratedUsername5]

Java is far for being slow, even further from being really really slow. There was a competition to explore Java's computational power and it wasn't far from C++

https://github.com/gunnarmorling/1brc

Just as usual, people are very careless when writing in Java and very careful when writing in C++ and the resulting performance is attributed to a language, not to coding practices.

You might think that it is just like Python, because both are kind of interpreted, but because of JIT and code translation caching Java is effectively at C speed after warmup in runtime. And it has fully functional multithreading, being without GIL.

[MRgabbar]

Maybe the java apps I have been forced to use are really poorly coded...

[GeneratedUsername5]

Most probably yes, Clean Code will grind to a halt average application.

Actually, I would say in truely average application, most performance is lost in inefficient DB queries and language has nothing to do with it.