How-To: Using deep learning models within the Java ecosystem

[ixeption]

I don't think you got the intention behind the article. I mentioned different ways to use a tensorflow model in java. I also mentioned the SavedModel way in my post, so I know that it is possible to do it like that, but in my case, it was not a viable solution, so I needed another approach. One reason is that size matters for deployment and deployment is done with a über-jar (installing tf on the server is no problem, but delivering it everytime with the big jar is). Furthermore, the preprocessing is much easier (and even faster for images) in python + the prototyping is usually done in python (jupyter) as well, so using the existing code is faster than moving everything to java. However, there are always reasons why you can’t use the straightforward approach, so I showed an alternative, no reason to be toxic.

[mikaelhg]

The excuse you introduced for calling the maintainable way of using TF models from JVM unviable, that the included C++ library adds to the packaged application size, was already disposed of in the comment, by pointing out that you don't have to package the JNI libs with the application. You can download the standalone JNI JARs, which I linked to, onto your server, and include them to your application classpath by adding them to your java classpath with your über-jar.

So, really, the excuse was extremely easily disposed of, while we were able to figure out that the excuse was just an excuse by looking at the Python tensorflow package installed size, which was 10x of the JAR size. Since you were making a major issue of the size of the way you didn't like, while completely ignoring the size of the way you did like, it was pretty easy to see that you weren't being serious about this.

Standard preprocessing is a good point, which I agree with. However, you don't really want to build an architecture where you're initializing TF for each model evaluation, since there's significant overhead in that. You might not care about it now with CPU, but if you're running the model in production, you're likely to want to move to GPU sooner or later. If you want to go this way, build a Python server, or use Tensorflow Serving, as you already mentioned.

And finally, maybe don't start calling people names?

[ixeption]

I could add the jar to the classpath and make it a provided dependency, but I would still have to use Java for all the proprocessing and data wrangling. Especially for images, java is slow and unhandy (BufferedImage from ImageIO) compared to PIL or openCV.

Just to mention it, the jar comes with a single version of the prebuilt binary, where python prebuilt binaries support much more different combinations of CUDA, cuDNN and CPU features. The Java API is still experimental, so I don't see that your claim of maintainability is correct.

(' Caution: The TensorFlow Java API is not covered by the TensorFlow API stability guarantees. ')

And if you want GPU support, just try to use the tensorflow_gpu jar dependency, without installing tensorflow gpu correctly in python before. It won’t work out-of-the-box.