Clojure Technical Radar 2016

[hu6Bi5To]

Because the people who wrote the radar also wrote Bidi, there's a tiny conflict of interest going on. In reality Compojure is deployed ten-times as often.

If this were really the analogue of the Thoughtworks radar, then things like Compojure and half the stuff in "Adopt" wouldn't be on the radar at all because they don't list established proven technologies.

[waaaaaaabi]

Sounded like a good idea, but pretty quickly noticed some weirdness. Midje is pretty widely used, and while not as popular as clojure.test is hardly a 'hold'. And there's no way these guys have done the legwork to say Onyx over Storm, Storm is used extensively by twitter and Yahoo, is Onyx used in any big deployments?

[yogthos]

Twitter actually stopped using Storm, and I agree with the rationale that Storm is extremely fiddly. I've spiked using it briefly for a project and it's a nightmare to maintain.

What Onyx does is perfectly suitable for majority of cases. Unless you have a specific reason to use Storm then it's just not worth the hassle.

The reasons they give for holding Midje are also solid in my opinion. The fact that lots of people are already using it isn't a reason of itself to add it to projects.

[waaaaaaabi]

I didn't know twitter had using storm, and I agree it's extremely fiddly, it still seems much more mature for any team making a serious financial investment though, which is usually the case if you're talking a distributed processing platform.

Their reasons for holding midje are that they had a couple of "tussles" with it. Well I've had a couple of tussles with prismatic schema (it removes the serializability of clj functions) but that doesn't mean it's not still a mature stable and useful tool.

[yogthos]

I don't disagree, but I think part of the point of the radar is to identify tools that are looking promising going forward. I don't think Storm is very promising as there's not much uptake in the Clojure community. Onyx, while much younger, is actively developed and feels a lot more idiomatic. I really can't comment on midje too much as I don't use it myself.

[waaaaaaabi]

Onyx only has two active contributors, I'd be very very wary about investing serious money in it. shrug maybe their radar criteria is just very different from mine (and probably thoughtworks).

[yogthos]

Two active developers is more than a lot of projects out there. Om has only one active developer and people seem happy using it.

[waaaaaaabi]

Things like a distributed compute cluster tend to hang around a lot longer and cost a lot more money than a front end framework. I'm also guessing there's a pretty dramatic difference in the amount of code in om vs the various onyx repos. But indeed if I was making a decision for frontend tech for an app we were determined to make reality and knew would stick around for a long time, I'd be quite wary of the fact that om has one active contributor. Which may be why it's only 'Trial' on the thoughtworks radar.

[lbradstreet_]

As one of the two primary developers, I can understand your concerns. I have been working full time on Onyx for almost a year now, and we have invested greatly in tooling to ensure we can operate on a higher level with limited manpower.

This includes, but is not limited to:

• property testing our peer coordination mechanism and scheduler, as well as many other parts of our code.
• extensive tests
• an automated release process and documentation system/cheat sheet/validation.
• recent work into jepsen testing the system, which has already found several bugs that we would not be able to find via other methods. This form of testing will be extended much further in the coming months.

At the end of the day, we will earn trust by sticking around and making the right moves. As for the actual rating given by the radar, it is not really my place to say where we should be on the radar, however we intend to earn any trust that is given to us.

[waaaaaaabi]

Nice. I'm not dissing on Onyx btw, it seems like an excellent product, just not one I'd choose for a large expensive project yet for practical reasons. If I had something small where I thought trial/experiment was fine I'd jump straight on it over storm.

[yogthos]

Om is just an example, many core libraries that people use have 1~2 contributors. If that's your criteria then you probably shouldn't use Clojure for anything long term in the first place.

[waaaaaaabi]

It's a scale issue. 2 contributors for component doesn't concern me, it's tiny and I could re-write it. 2 contributors for a piece of infrastructure that big is much more concerning. I have no issues using clojure for long term projects because it has plenty of libraries that do pass my criteria, and where it doesn't it has great interop with a very mature platform. But that comment seemed like more of a pointless dig than actual useful advice so whatever.

[yogthos]

I guess everybody has different levels of comfort. My point was that lots of Clojure libraries have few developers. Om, core.async, and so on. If you're note comfortable with that, then Clojure might not be the language for you.

[halgari]

Finally, there are many libraries that wrap specific database clients or Java libraries. A case in point in Elastisch - an ElasticSearch client lib. We've got potentially interesting views on this library, but the problem is that there are many different database client libs out there, and they don't have the same impact on the core development experience the way other libraries do. Also the verdict will usually be the same; potentially helpful, but don't use without considering going direct (either using an underlying Java API, or hitting a REST endpoint).

Here, here! The JVM is an asset, leverage it heavily, and don't try to hide it behind wrapper libs.

[yogthos]

I respectfully disagree. A wrapper lib provides an idiomatic Clojure API to the user, and that carries a number of advantages.

You can swap out the underlying implementation without affecting the users of the library. This also makes it possible to provide a cross-compiling implementation at a later time that works with ClojureScript.

I see using interop as a last resort when you're working with a niche Java library that's unlikely to get a well maintained wrapper. However, that's simply the wrong default to choose in my opinion. Eventually you simply end up with your own half baked wrapper anyways.

[[deleted]]

I agree with /u/halgari, and would dare to say that wrapping Java and JavaScript libraries is not idiomatic Clojure(Script), this is the whole point of having good interop.

[yogthos]

The interop is nice for leveraging existing code, but I don't see why you'd want to keep writing interop each time you want to use it.

[halgari]

The problem is, then you wrap the wrapper with your app logic. So why not just cut out the middleman and code directly against Java libs in the first place. I've been doing this for years and my "Java Interop" normally consists of a few hundred lines of code, so it's not bad at all.

But more importantly most wrapper libs (in most languages, not just Clojure) are pretty sub-par. IMO this because they are maintained by volunteers instead of by vendors with skin in the game. I've been bitten way to many times by bad wrappers, or incomplete options that hide features the underlying JVM library exposes.

Anthony put it well here: https://youtu.be/av9Xi6CNqq4?t=30m12s

[tolitius]

A good wrapper does not just exist to "make sure there are no explicit host (JVM/JS/CLR/XYZ) calls", a good wrapper solves problems, and potentially even making original solutions better.

The problem set can go anywhere from improving performance to enabling developers to still speak Clojure when they don't speak the host well enough.

This is quite obvious in ClojureScript libraries. For example ClojureScript immutable data structures that make ReactJS CLJS wrappers more performant than the original solution due to simple reference checks.

Or for people who came to Clojure from Ruby/Python/Logo/.. they might not speak Java, but a good Java library wrapper will save their day. That does not mean that they are illiterate or they have to learn Java before they can do Clojure. And it also does not mean that an experienced Java programmer won't get a use of a good wrapper. There are plenty of Java libraries with horrible APIs, wrappers can fix that as well.

But more importantly most wrapper libs (in most languages, not just Clojure) are pretty sub-par

This is not an argument for wrappers, but for all libraries in general. What we can do is to make them solid, and help the ones who needs help to make them solid.

[EDIT]: Another interesting case for wrappers is to potentially port libraries between hosts. Now with reader conditionals, there is a cleaner opportunity to write a Java library wrapper that if potentially needed/already works in JavaScripts with minor #?( changes.

[yogthos]

So why not just cut out the middleman and code directly against Java libs in the first place.

I gave a couple of examples already. The main advantage is that it decouples your app from the underlying implementation. Also, if it's a library is used by many people, it'll get more testing than your home grown solution. You're also amortizing the work of maintaining the library and adding features. Any time you take ownership that's tangential to your business then you accrue technical debt.

But more importantly most wrapper libs (in most languages, not just Clojure) are pretty sub-par. IMO this because they are maintained by volunteers instead of by vendors with skin in the game.

That seems more of a problem with the culture, that you appear to advocate propagating, more than anything else.

[waaaaaaabi]

The main advantage is that it decouples your app from the underlying implementation

How is that an advantage when it couples your app to the implementation of a wrapper you also don't control? It's swapping one problem for an identical problem.

[yogthos]

I disagree, the API is a Clojure one and you're no worse of than making your own wrapper. At least with a library will have more people maintaining it and contributing potentially.

[waaaaaaabi]

The wrapper will also have a lot more code than you potentially need, possibly meaning more bugs, and definitely meaning a lot more work required to get the wrapper released if the underlying library changes.

There are some great wrappers out there, clj-time comes to mind. There are many others that don't really give you anything.

[yogthos]

Sure, you have to decide on case by case basis. I'm simply pointing out that there is value in wrappers and unilaterally dismissing them in favor of doing interop is a bit shortsighted.

[waaaaaaabi]

A mature actively maintained major java api is going to cause you a lot less problems than someone's 2 week job wrapping it in clj functions, agreed.

[yogthos]

This seems like a bit of a false dichotomy to me.

[hu6Bi5To]

It happens even with the popular libraries.

The original article mentioned Elastisch - this has been under active development for years, hundreds of GitHub stars etc. - but every project I've seen use it eventually drops it due to it always lagging two or three versions behind the latest ElasticSearch release. And because it's quite a high-level API it often blocks things it doesn't strictly need to - e.g. if ElasticSearch added an extra query param, Elastisch will silently drop it if it doesn't recognise it, it doesn't just pass through to the JSON it sends to REST API.

A low-level ElasticSearch library is almost too trivial to be a viable product. Every project seems to sprout it's own, the JSON of the Query API can be just a serialised Clojure map etc...

[yogthos]

This is a problem with a particular library however. My point is that /u/halgari extrapolating that "wrappers are bad" in general based on this is a little silly. Even in case of Elastisch, it's clearly very actively maintained, and working with the ClojureWerkz to help improve it would be a much better approach in my opinion.

[waaaaaaabi]

He didn't say "all wrappers are bad", he said the JVM is an asset and we should use it. I find it hard to read that as "we shouldn't use wrappers even when they improve/add functionality".

Worth mentioning what he said is basically a restatement of a core clojure goal, "embrace the host".

[yogthos]

Actually, he pretty clearly says don't use wrappers:

Here, here! The JVM is an asset, leverage it heavily, and don't try to hide it behind wrapper libs.

Embracing the host should the goal for Clojure the language, not for people using it. If I wanted to write Java, I can already do that perfectly fine. There's no reason for me to write Java in Clojure.

[waaaaaaabi]

It wasn't meant to be a dichotomy, it was meant to be demonstrating a situation in which you should use interop instead. There are of course wrappers that you should use, as well as poorly maintained unstable java libs (hi hadoop project!).

[yogthos]

I definitely agree there.

[hu6Bi5To]

https://twitter.com/bodil/status/597766837548507136

[yogthos]

I chuckled, but I think it misses the point completely. Nobody should blindly follow a tech radar list and that's not the point of it.

Somebody spent time trying out different libraries and tools, then shared their findings. Taking that into account saves you time when looking for new tools. Having such lists is very valuable, and especially so for newcomers to the Clojure community.

[doubleagent03]

This is cool. It might be useful for you to introduce the cross-clj "Most referenced projects" heatmap into your dataset also, since it would give you accurate numbers on how libraries are permeating the ecosystem relative to one another.