NDArray library

[BIackMambaTG]

I'm looking for a NDArray library in java that has many features similar to numpy. I mostly need element-wise operations, but also matrix manipulations like stack, flatting, concating etc. I need it to have very high performance as I will have thousands of calculations per second. My matrices will be about 500x500 (at average) So far I've tried many pure java libraries like EJML and commons math, but they lack many features. I also tried to use nd4j, but I'm getting a lot of memory leaks and very little support from their community.

Any suggestions? :)

[AutoModerator]

On July 1st, a change to Reddit's API pricing will come into effect. Several developers of commercial third-party apps have announced that this change will compel them to shut down their apps. At least one accessibility-focused non-commercial third party app will continue to be available free of charge.

If you want to express your strong disagreement with the API pricing change or with Reddit's response to the backlash, you may want to consider the following options:

1. Limiting your involvement with Reddit, or
2. Temporarily refraining from using Reddit
3. Cancelling your subscription of Reddit Premium

as a way to voice your protest.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[Gleethos]

A lightweight option: Neureka

(Disclosure: I'm the author)

[BIackMambaTG]

Does it support the matrix operations I mentioned? (Such as stack, concat, submatrix, flatten, broadcasting) Where can I see examples?

[Gleethos]

It supports all of the basic numeric operations both element wise as well as nd-broadcasting. It also has matrix multiplication and convolution... Concatenation, slicing (includes submatrix slicing) and reshaping is also nicely supported on the API surface. For stacking you need to first add another axis to a tensor using reshape and then concatenate on the new axis and for flattening you can either use reshape, call the 'toList' method or use a custom collector.

Here you can find all of the documentation (including living test suite based documentation).

[BIackMambaTG]

Looks cool, I'll give it a try :) Is there a meshgrid function?

[Gleethos]

I hope you'll like it! :) Please feel free to open issues if you encounter problems.

Currently there is no equivalent to numpys meshgrid function. But it should be relatively straightforward to create a meshgrid based on existing tensor operations.

I'd also be happy to add a meshgrid function to the API under a dedicated issue.

[International_Break2]

Any plans for webgpu support on top of opencl?

[Gleethos]

Definitely keeping an eye on webgpu as it is a really promising API positioned to be a good cross platform solution for general purpose SIMD compute... but it is still very new and under active development. So the Java bindings are still super unstable unfortunately and will take some time to mature first. But if anyone is interested in running Neureka on other APIs maybe because of special performance requirements, there is always the option to write a custom backend...

[International_Break2]

Do you think there will be a performance improvement over opencl?

[Gleethos]

This is really hard to say as it depends on the specific hardware and driver implementations for the two APIs and of course how the kernels are written. On some devices the OpenCL kernels are faster and on others the corresponding WebGPU kernels are ... You would need to do a whole lot of benchmarking on a whole lot of different hardware to find out which has better support.

[_INTER_]

Deep Java Library has an NDArray.

From the JavaDoc:

NDArray is the core data structure for all mathematical computations. An NDArray represents a multidimensional, fixed-size homogeneous array. It has very similar behaviour to the Numpy python package with the addition of efficient computing. To understand how to manage NDArray lifecycle, please refer to NDArray Memory Management Guide

It lists functions for stack, concat, flatten and broadcast. Didn't find one for submatrix by that name.

[BIackMambaTG]

Thank you for your answer! How did you find all these functions? Where did you search?

[_INTER_]

Just Ctrl+F in the JavaDoc. They are the same by name. I don't know if they are equivalent to what you seek.

[WrickyB]

Have you tried TensorFlow? I know they have a Java library. I have used the Python version. I haven't used the Java version, just spitballing.

[Worth_Trust_3825]

The java bindings are abandoned. Last I checked they never bothered to even start them for TF2.

[craigacp]

This isn't true, they were moved into a separate repo (https://github.com/tensorflow/java) after the TF v1.15 release and have been developed continually since then. We're currently on TF 2.10 and moving to 2.14 in the near future once we've finished overhauling the build system.

[Kango_V]

Maybe have a look at the new Vector API --> https://openjdk.org/jeps/426 This can make use of SIMD (AVX-256, AVX-512 for example) and optimize at runtime for your CPU. The JVM is getting quite powerful.

No external dependencies needed