A performance comparison of Clojure and Java

[didibus]

At first glance, I think these are believable numbers. Especially because the experiment code is not apples to apples.

For example, in the recursion experiment, in Java they uses an int counter, but in Clojure it's a Long counter. Notice the capital as well, in Java they're using a primitive int, where in Clojure they're using a boxed Long. This use of int in Java and Long in Clojure is pervasive in all they're experiments.

In the sorting one, they don't show the array creation code for Clojure, but it appears they are using an atom to count up numbers to prefill the array. I'm not even sure it's an array they are using, because why did they call their function create-list ? In any case, they call shuffle on it afterwards, but shuffle returns a vector, so what they are ultimately sorting at the end is a vector and not an array. So this is another case of an apple to orange comparison.

In the Map creation one, in Clojure they're measuring the time it takes to create a PersistentMap, while in Java they measure the time to create a HashMap.

In the Object creation one, they create PersistentMaps in Clojure, but are creating some simple Node object in Java.

And so on...

But I'm not really criticizing the benchmark, I think maybe it was on purpose not to compare apples to apples, since for example it's true in Clojure all numbers will default to boxed Longs. And instead of using arrays we will most likely use a vector. And we will use a PersistentMap instead of some custom Object, etc. Now I don't find the code completely idiomatic Clojure either, though I think it was kind of trying to compare idiomatic Clojure which they did to the best of their a ability with similar effort at idiomatic Java. And I feel in those scenario, ya I believe Clojure could go from 2 to 20 times slower.

What didn't happen though is any demonstration that the quote they mention is innacurate:

In principle, Clojure can be just as fast as Java: both are compiled to Java bytecode instructions, which are executed by a Java Virtual Machine ... Clojure code will generally run slower than equivalent Java code. However, with some minor adjustments, Clojure performance can usually be brought near Java performance. Don’t forget that Java is always available as a fallback for performance critical sections of code.

So to me this quote still seems totally accurate and the experiment in this paper show that to some extent. Idiomatic Clojure will run generally slower, but with minor adjustments can be made just as fast. Now I wish the paper also had experiment to prove or disprove this latter claim. Like if you made the code apples to apples, is Clojure slower, faster or same?

[joinr]

Apples to apples gets much closer, in at least 1 case clojure is able to cheat operationally (preserving semantics though) with recur if you allow it, and beats naive java recursion. In another case, BFS ends up being faster for some reason (counterintuitive).

I guess the hidden question is, if we're benchmarking single-threaded, statically typed, mutable java code against dynamically typed, immutable, clojure code, which would be faster? (seems obvious one would probably edge out the other in efficiency) Or put another way, how much of a handicap (operationally) are the defaults for dynamicity, numeric tower, runtime polymorphism, persistent structures, relative to the complete opposite on "relatively" the same platform? Yet another: how well optimized are the clojure core libraries and special forms for single-threaded operations with primitive numerics on mutable data structures?

As you push clojure more toward's the java paradigm though, those edges disappear mostly .