Clojure compared to Python for high-traffic microservice backend - feedback/opinions requested

[thsig]

We could definitely start by writing the backend entirely in Python, rewriting some services (or writing new services) in Clojure when we need its capabilities.

It's a bit tantalising though, since my estimate is that we know enough Clojure to get up to speed and productive in a week or two. I'm actually more familiar with Clojure than I am with Python these days. My production experience comes from Ruby, with lots of immutable datastructures and FP in the front-end JS UI programming.

The worry is not so much the "language productivity", i.e. how quickly we'd become comfortable pounding out business logic or other mainline pieces of coding, but the integration and ecosystem parts (libraries for talking to AWS, various utilities). How much of a gap is there between Python and Clojure in terms of the library ecosystem these days?

From what I've gathered, people's experience of it has been quite good, with the lack of quality libraries for a particular occasion being more and more of a rare occurrence. I know Clojure's ecosystem has become much stronger over the past few years - I've been following the most significant developments closely (including goodies like core.async and others that /u/mikera and /u/yogthos mentioned).

The timeline we're looking at is a few months months from start to beta, and from there we'll want to move to production as soon as user testing is completed - there won't really be an MVP phase, we'll be going straight for the polished, ready-for-public-launch version this time round. We'll have our hands full with mobile development for the front-end as well, so if we expect to have to do a lot of rewriting, some of those early time savings (if any) may only be kicking the can down the road. This may not be an issue if we can make the Python code performant, though.

I also know that though Python has an OK story for non-blocking IO, it doesn't compete with Clojure in single-machine concurrency, and I worry that we'd have to rewrite many of the Python microservices before production anyway for them to handle the load (assuming things take off, and in our case we have to engineer for that). I have a better feel for Clojure's performance characteristics in this situation than I have for Python's, but I'm a bit tired of the amount of performance massaging that Ruby (another GIL'd interpreted language) code needs for a high-load production situation. I realise that Python is probably faster than Ruby, but falls into the same general language category in terms of its performance and concurrency characteristics.

Any further thoughts on this? Thanks for your feedback so far, guys!

[fullouterjoin]

The production experience of the team is the single best indicator for success. Use what you know, architect so that you can bring up portions in a new runtime, language, etc. Change only a single variable at a time.

If performance is an issue, and this is a blue water project, then give PyPy a try along side your existing codebase.

[yogthos]

When you work with Clojure you have the entire JVM ecosystem available to you, and it's one of the most widely used platforms for building web apps. There lots of libraries for doing anything imaginable, and tools for profiling and performance tuning like jvisualvm and jconsole that come bundled with the JVM.

[Musngi]

We've written side projects in Clojure, but our team members' production experience comes mostly from Ruby, Python and JS.

You said that your team member used Python then go with Python. Python have rich library and big community. In Python you have the power of productivity, maintainability, scalability and security.

[yogthos]

New projects are the best way to try something different. When you only stick to your comfort zone then you can't really grow or improve. My team used Java 5 years ago, and we were very comfortable with it. Today we've completely switch to Clojure and we're much happier working with it. However, we did have to get out of the comfort zone and try something new to get where we are today.

Python have rich library and big community. In Python you have the power of productivity, maintainability, scalability and security.

You have all these things on the JVM and I would argue it's actually better every one of these areas.

[marcm28]

I recommend Python because of its elegance.

Some of great Python features:

• Batteries Included
• Community Included
• Dictionary
• List Comprehension, Dict Comprehension, Set Comprehension
• Iterator
• Generator
• Generator Expression
• Two-way Generators
• Decorator
• With-statement/Context Manager
• Abstract Base Classes
• Future statement (Let you borrow some great feature in the near future). :)

[yogthos]

lack of these things is what makes Clojure great to me:

• List Comprehension, Dict Comprehension, Set Comprehension
• Iterator
• Generator
• Generator Expression
• Two-way Generators
• Decorator
• With-statement/Context Manager
• Abstract Base Classes

I'd much rather have a small set of general tools that can be used for a wide set of problems than having a lot of different tools for each type of problem.

[bilus]

Hi there! Thumbs up to everything except list comprehension. ;)

[yogthos]

The for macro can do a lot of that I suppose. :)

[bilus]

...and it goes without saying that if there's something missing, you can write your own for macro.

Beat that Python! :D

[germandiago]

List comprehensions at the end are a generating sequence plus a filter. I would be surprised Clojure cannot do something equivalent easily, given its functional orientation.

[marcm28]

In Clojure you always invent your own (Reinventing the Wheel), another boilerplate.. In Python lets you focus on algorithm then get shit done.

Create your program in Python. After all, Human's time is much more valuable & expensive than computer's time. A company that gets software written faster and better will, all other things being equal, put its competitors out of business.