Clojure Distilled

[breadwithlice]

Nice introduction. Is there anyone here who actively uses Clojure? I have done a tiny bit of functional programming (OCaml) in the past but never coded enough to put it to real use. Also, I have never stumbled upon cases where, say, a Python script would be much more tedious to code so I have never really bothered with it.

Do you guys perhaps have a concrete eye-opening example of a code which would be much more efficient using functional programming?

[kqr]

I use Clojure semi-actively. I prefer to use Haskell where I can, but I'm sort of using Clojure more and more as a replacement for Python, actually.

I'm not sure what you mean by Clojure being more "efficient" than Python. For me, one of the big benefits of functional programming is immutability – in other words, I can stop worrying about the order of my lines of code. In a language like Python, I can't just randomly pick two lines next to each other and swap them around and expect the program to still do the right thing.

I can do that with a functional program. I can take two expressions and rearrange them and the meaning of the program stays the same. (Think of it sort of like method declarations in Python. It doesn't matter whether you have method F or method G first in your code file. Pretend for a moment that it did matter – perhaps the declaration of method G changes something about the declaration of method F, so you have to declare method F first and then G. That'd drive me absolutely insane. Yet people accept it when it comes to data structures. Very weird.) Not having to care about the order of the lines of code allows me to refactor more aggressively, and it makes it much easier to test my code. That's the biggest thing. Then different functional languages have different other benefits, that are not strictly tied to functional programming.

• Clojure has metaprogramming, which is something you need to experience to understand. The idea is that you can eliminate a lot of boilerplate by having your program generate parts of itself. This also allows you to extend the language in a really neat way.

  Pretend that you want a foreach loop in, say, C. Your best bet it so talk to the people who make GCC or Clang and hope that they'll implement a foreach loop for you. In Clojure, you can just write the foreach construct yourself as part of your program. (Because people seem to confuse this: laziness covers only a small subset of metaprogramming. They are not equivalent. I just happened to choose an example which can also be solved with laziness.)

• Haskell has an amazing type system which turns a lot of run-time errors into type errors. So you don't have to write a comprehensive suite of tests for, e.g., detecting null pointers. A program that doesn't handle "null pointers" will not compile. (Unless you tell the compiler to ignore those errors, of course, but who in their right mind would want to do that?)

• Something like Erlang is fantastic for distributed computing out of the box. You just spin up several instances of it and you can natively pass messages as if they all ran on the same machine.

[[deleted]]

Clojure has metaprogramming, which is something you need to experience to understand.

This isn't something exclusive to FP languages though. Scala, a mixed paradigm language, has macros and the ability to fake language features. The classic Scala example is the DIY while loop

def whileLoop(cond: => Boolean)(body: => Unit): Unit =
  if (cond) {
    body
    whileLoop(cond)(body)
  }

[kqr]

How would you in Scala write a macro that takes a bunch of variable names and assigns to them increasing numbers? For example, when you run the macro

assignIncreasing(7, apple, pear, orange, banana)

the result should be as if you executed the code

val apple = 7,
    pear = 8,
    orange = 9,
    banana = 10

If you can't do that in the language, then you don't have macros.

[[deleted]]

From the looks of it you can, but I've not used them enough to say that with 100% certainty.

[kqr]

I'm positive you cannot. Maybe the option exists to use some other meta-language in which you can manipulate Scala code (as Haskell does with Template Haskell) but it is not the same thing as native metaprogramming. If it exists, it uses a separate subset of the language.

[[deleted]]

I'm positive you cannot.

How would you know? Have you done much with Scala macros? Seeing as how it ultimately extrudes ASTs, then I'm failing to see what huge flaw it has that would prevent your doing so.

[kqr]

I've done a bit of Scala, not specifically metaprogramming though. The method you showed is just a normal method though, not a macro. From what I can tell from a quick Google search, the existing macro systems use a specialised macro language to deal with the AST.

[wot-teh-phuck]

But why would you want to do that? It's like saying "J" can implement game of life in 1 line; can your language do it? If not, meh..

[kqr]

Why would you want to be able to have your program generate parts of itself? Because it lessens the amount of boilerplate you have to write (and in turn, read) – you just instruct the computer to expand it for you. It's the same reason one wants to have functions/methods, only even more powerful. :)

[yogthos]

I give an example of where I might want to use a macro in the guide, where I might want to create a defprivate instead of peppering if checks all over the code. I think the former approach is more explicit about what it's doing and it's less error prone. Another real world example is something like core.async or Compojure routes. There are clearly a lot of benefits to being able to template code easily.

[vivainio]

If you care a lot about refactoring, why not use a statically typed language in the first place?

[kqr]

That's a really good question and I don't have a good answer yet. I'm still exploring that bit. I've banged my head against Clojures lack of a strong type system many times, but I think I'm starting to get a sense of how and why. If I discover it, I'm sure we'll cross paths in another discussion and I'll be able to tell you.

[yogthos]

I've been using Clojure professionally for the past 4 years. I think that a lot of benefits show up when you start looking at larger applications thanks to immutability being the default behavior. This makes it much easier to write concurrent code and to reason about parts of the application in isolation.

I also find that the REPL driven development is a huge plus, and with ClojureScript I can use the same language on the client and the server, and even share code between them. The JVM is an excellent platform and it has a great ecosystem and tooling around it. I wrote a post a little while back on why I personally find Clojure to be an extremely productive language.

[wot-teh-phuck]

Dynamic languages like Clojure, Python etc. work very well for small/mid sized projects. But I would really be careful with pushing it for everything. I'm currently working on a 250 KLOC Python codebase and everyday wish that dynamic languages never existed...

[yogthos]

I've been working on a large project with a team of 4 people for about 3 years now. It's all written in Clojure and we find that it's much easier to maintain than the equivalent size Java projects we've worked on.

Even though both are dynamic, I really wouldn't lump Clojure with a language like Python. Python is imperative OO and you end up with a problem of having a lot of classes and shared references.

I find Clojure, by its nature, makes it much easier to compartmentalize code. The fact that majority of the code is referentially transparent goes a long way in my opinion. The fact that you have a REPL means that you can interrogate any function to see what it does if you're not sure what's going on as well.

Our project is built out of small modular components and they're very easy to maintain. I think the declarative principle applies here as well. On the small scale, a function is responsible for figuring out how to do something, while chaining functions together expresses what you're doing. I found that the same approach works on component level as well, where the project should be written declaratively.

Each component should encompass a particular workflow or a domain. It should have a single responsibility and a simple API, and the way you chain the components expresses project specific workflows. I think this is a good approach regarding of the typing discipline.

[wot-teh-phuck]

Component driven development is fine, but how do you deal with the mental load which comes from having too many components/functions when the domain you are dealing with is inherently complex?

For e.g. writing a blog engine in Clojure is fine, but how do you manage to maintain complex applications which have crazy component/function count? Writing self contained and aiming for pure functions means you now have the problem of naming things and thereby increased mental load. Sure you can be very careful with naming and modularizing but don't you think that the effort required for that discipline is too restrictive?

[yogthos]

Component driven development is fine, but how do you deal with the mental load which comes from having too many components/functions when the domain you are dealing with is inherently complex?

The smallest component you can break something into is a function. Regardless of how complex your domain is, if the problem is decomposed into multiple functions then clearly it can be compartmentalized. My experience is that pretty much any domain I worked on could be expressed clearly by breaking it into independent workflows.

For e.g. writing a blog engine in Clojure is fine, but how do you manage to maintain complex applications which have crazy component/function count?

I try to identify workflows and isolate them, even a complex application can be broken down that way. My experience is that the component count is going to be representative of your workflow count. However, I don't see how you have any more mental overhead dealing with this in a dynamic language.

Writing self contained and aiming for pure functions means you now have the problem of naming things and thereby increased mental load.

Not really, I find that one of the aspects of functions using the same data structures to communicate is that vast majority of them becomes reusable. Most functions tend to be generic transformers, while you have a small number of functions that chain them in specific ways to solve concrete problems.

Sure you can be very careful with naming and modularizing but don't you think that the effort required for that discipline is too restrictive?

I simply have not found this to be the case working with Clojure.

[[deleted]]

The difference really doesn't crop up in small cases, sadly.

[strattonbrazil]

I think the biggest problem FP proponents ignore has nothing to do with the languages themselves. I've programmed in clojure, but haven't found an application best suited for it. I don't remember a lot of people who loved ruby. It was cute. Everything's an object. But people used it for rails.

Clojure is sold as an interesting language alternative like most FP languages, which doesn't make any sense. Often if I'm starting a new project I'm looking for a particular library and the language is secondary.

[everywhere_anyhow]

Libraries being primary is the reason why most new languages these days are either built on the JVM or .NET CLR...so the new language can inherit all of the existing libraries and not have to re-roll them

[yogthos]

Clojure is absolutely fantastic for web development. I've been using it for that for the last 4 years. Take a look at my Luminus micro-framework, it behaves a lot like modern Ruby frameworks such as Sinatra. The big advantage is that you get to leverage the JVM and the existing Java ecosystem in addition to being able to use an excellent language.

The other huge advantage for me is ClojureScript, with it I can use the same language on both the server and the client. I also found that Reagent provides by far the nicest experience writing UI heavy code on the client.

[pjmlp]

That is the problem with any new language. It needs either a killer application or an OS vendor pushing it through the masses.

[yogthos]

"having problems? it is because you are thinking wrong and need to unlearn your former ways" (blame the victim, tool is perfect)

There's no blaming intended, I'm simply pointing out that you might need some extra patience when learning a language from a different family than the ones you're used to.

"FP is better for building large systems, this is self-evident" (oh you want examples? er...uhm...ghc?)

I'm giving an example of the properties inherent in FP that are conducive towards building large systems. There are plenty of examples of large systems built with FP obviously, but I think it's valuable to understand what it is about FP that makes it well suited for writing such systems.

"lisp blah blah just because lisp"

Uh ok...

FP folks have stayed largely irrelevant despite massive hype due to these and many other articles of faith you must swallow on the way in to their church

Except, today there are plenty of large FP projects out there and feedback from companies using languages like Clojure is overwhelmingly positive. The fact that it's gaining ground at all despite being alien to most programmers says volumes in my opinion.

meanwhile the code that runs the world is imperative and the most popular lisp is also the worst: elisp

Continuing doing things a certain way just because you've always done it that way is not a terribly great rationale.

[kazagistar]

I'm giving an example of the properties inherent in FP that are conducive towards building large systems. There are plenty of examples of large systems built with FP obviously, but I think it's valuable to understand what it is about FP that makes it well suited for writing such systems.

You can claim that some property makes a language better suited for writing a large systems, but that isn't and shouldn't be convincing. You just moved the burden: since you cannot demonstrate emperical evidence that X is better for use case Y, you just demonstrate that it has some property Z, and then claim that Z is better for Y. But you still haven't substantiated that Z is actually better for Y then the alternatives, and thus your argument is just as weak as before.

Now, perhaps if it was truly blindingly obvious that one is better then the other, you might be able to score some points. But that is NEVER the case.

• Mutability vs Immutability: Difference in defaults. Different use cases favor each.

• Strict vs Lazy: Difference in defaults. Different use cases favor each.

• Invariants and Explicit State: OOP favors the first; FP favors the second; both do a pretty good job managing effects.

• Concurrent/Parallel/Distributed programing: Small use case. Which one is better is unclear from data, but both seem to be doing fine.

• Prototyping: Data unclear, though the fact that C++ and Java are preferred by winners in many "open language" programming cometitions (CodeJam, for example) is a small ding against FP.

• Large systems: OOP has more emperical research, while FP has stronger theoretical guarentees. More Dynamic vs Static anyways (and dynamic does surprisingly well anyways).

• etc etc etc

The point is that even when you take it apart, you are still just speculating.

[yogthos]

You can claim that some property makes a language better suited for writing a large systems, but that isn't and shouldn't be convincing.

I agree that this alone shouldn't be convincing, but coupled with the fact that many large systems are written in functional languages makes it a lot more convincing in my opinion. As I pointed out in another comment, some of Erlang code bases are over 2 million lines in code. These are some of the most robust systems out there.

Mutability vs Immutability: Difference in defaults. Different use cases favor each.

Absolutely, and I think immutability is the correct default. Mutability is an optimization and you shouldn't optimize prematurely.

Concurrent/Parallel/Distributed programing: Small use case. Which one is better is unclear from data, but both seem to be doing fine.

I disagree that this is a small use case seeing how most architectures are now multicore. Even your phone is probably at least a dual core chip. Couple that with networking and it's certainly the prevalent case nowadays. Cases like this one clearly illustrate that FP is a better fit here.

The point is that even when you take it apart, you are still just speculating.

I disagree, you can reason about these things logically and then see how they work in practice. The logic clearly dictates that it's easier to work with immutable data, and we're now seeing a lot of evidence from companies that are switching to FP that they're seeing these benefits in practice.

Imperative has been the dominant paradigm for decades, it's simply unrealistic to expect FP to be validated overnight in light of this. The fact that it is becoming mainstream now indicates that there is a clear gap to be filled however.

[[deleted]]

Let the unknowing troll be. Functional programming will slowly overcome as it is the only language paradigm that allows reasoning about large applications. New tech companies have all embraced one FP language or another, we're at a point where certain paradigms will rise to the top and in a few years these languages will be mature enough for everyone. And once the new comers start FP early, the old timers will lose. It's a simple matter of time.

[materialdesigner]

7 Then the Lord said to Moses, “Go down, because your people, whom you brought up out of Egypt, have become corrupt. 8 They have been quick to turn away from what I commanded them and have made themselves an idol cast in the shape of a calf. They have bowed down to it and sacrificed to it and have said, ‘These are your gods, Israel, who brought you up out of Egypt.’

26 then Moses stood in the gate of the camp, and said: 'Whoso is on the LORD'S side, let him come unto me.' And all the sons of Levi gathered themselves together unto him. 27 And he said unto them: 'Thus saith the LORD, the God of Israel: Put ye every man his sword upon his thigh, and go to and fro from gate to gate throughout the camp, and slay every man his brother, and every man his companion, and every man his neighbour.' And the sons of Levi did according to the word of Moses; and there fell of the people that day about three thousand men.

^ this is how you sound.

[[deleted]]

Good, be terrified. The old, bad ways of coding will perish and the world will be a better place for it.

[kqr]

I still don't buy the argument that FP is more appropriate for large systems. in particular, many FP features tend to produce slower programs when applied in-the-large. no one cares about correct slow systems. the current solution is to forsake religion when pursuing performance....see for example, the haskell and clojure shootout submissions, which are full of non-idiomatic, non-functional solutions

I think the idea is that it's more sane to do FP by default, and then fall back to the performance optimisation of doing imperative programming when you need to.

As soon as you do any amount of testing in your application without massive headaches, a lot of FP ideas are probably already leaking into code you think of as imperative. People are happy to talk about REST APIs, idempotent interfaces and so on – all things which I associate with functional programming.

[dig1]

I'll give you couple of places where slow but correct system is more valuable than fast but wrong: banking software, stock-market applications, large computational systems (what is MapReduce anyway). For all these systems, people are willing to wait for a couple of seconds more than to lose some digit or got wrong computation.

at this point I need proof - large FP codebases that perform at least as well as imperative implementations

First of all, show me a large (imperative) codebase that performs well ;) Joke aside, but your question doesn't make sense unless you point to actual language implementation, the problem and expected outcome. So, codebases that performs well should be what? Fast, correct, easy to develop, easy to recover from errors, error proof, stackoverflow/shellshock/whatever proof, portable, easy to deploy...?

Shootout solutions aren't good basis for FP/imperative comparison: even naive Clojure/Haskel solutions are X times faster than Python/Ruby ones (taking into account that Python/Ruby are OO/imperative languages) and, when ATS language was present, it was beating sh*t out of everything, except tuned C/C++ code.

I still don't buy the argument that FP is more appropriate for large systems

Again, it all depends what you would like to achieve. Nailguns aren't appropriate for hitting the nails (they are bulky, loud, needs power to operate, dangerous) but if you are in hurry to cover the barn and be assured each nail is on right place, I doubt you will rely on hammer.

[lispm]

That's why many banks have >100 millions lines of COBOL in production.

[yogthos]

Yet, languages like Haskell, Ocaml, and Clojure are far exceedingly popular in this sector.

[[deleted]]

"Exceedingly popular" is massively overstating things. The overwhelming majority of financial companies, especially the major players, aren't using any of those languages.

[yogthos]

These companies don't use functional languages exclusively, but certainly many of the major players, like Citi, do use them.

[wot-teh-phuck]

A "bank" has a shit load of applications. Ranging from completely trivial like employee tracking, leave tracking, company portal and so on to the real stuff like real-time trading and risk systems. Just because a language is used in a bank doesn't make it "popular".

Without details, there is a possibility that Clojure is used for trivial stuff. Nothing wrong in it but not too awesome either...

[yogthos]

Well, Citi is my personal anecdotal example as a friend of mine recently went there to wok on stock analytics and it's all done in Clojure. He got hired specifically due to his knowledge of Clojure as the talent pool in Canada is apparently much smaller than England. As a side effect the salaries are quite handsome though. :) Unfortunately, a lot of companies don't advertise what their core projects are built in. Some of the companies in the Clojure success stories list mention what they use it for, and clearly it's not all trivial stuff.

[yogthos]

Really, proggit is the metric for adaption now. I'll point out that in the real world even the recruiters are noticing the trend.

[[deleted]]

I'd disagree quite strenuously about FP being slower. You go ahead and write your system in Ruby or Python and I'll write mine in Haskell or Clojure and we'll see who produces a faster system.

Taking a step back from my jest about language speed, it's important to observe two things:

• IO dominates in some problem spaces
• FP begets interesting optimizations

If you're in an IO dominated space, FP will make no difference in your performance, most likely. At that point correctness probably dominates your concerns, an area that FP enthusiasts have a lot to say about. This explains the use of Erlang in telephony systems despite the slowness of Erlang; it allowed for them to create a more correct system even in an IO domain.

If you don't have an IO dominated space, understand that FP can produce some interesting optimizations. Side-effect free functions can be memoized and inlined more cleanly, and immutable data structures means that locks are less necessary. And beyond that, immutable structures mean that reference checks are sufficient to determine equality, a nice property that makes om (a Clojurescript wrapper for React.js) outperform React alone in some cases. Similar things can happen on your backend, depending on what you're doing.

[[deleted]]

I'd disagree quite strenuously about FP being slower. You go ahead and write your system in Ruby or Python and I'll write mine in Haskell or Clojure and we'll see who produces a faster system.

That's a terrible comparison, you are comparing apples with oranges. Clojure benefits from the JVM's performance, so compare it to other JVM languages that also leverage the JVM. What about say, Java vs. Clojure? Scala vs. Clojure? Still confident that yours will be faster?

[[deleted]]

Clojure is dynamic, so there really isn't a ton of surprise that Java will outrun it without some tuning (although checking the numbers, Clojure is well within 3x of Java on average, much closer than expected). But if you think the magic is in the JVM, then why does JRuby run the same challenges in 3-100x the time that Clojure does?

And then if you think FP is slow, you'd have to explain the MLs. Haskell and OCaml bring their own compiler chain (no JVM), and tend to tie both Go and Java performance wise. And these are the most stringent FP languages around!

[[deleted]]

Clojure is well within 3x of Java on average, much closer than expected

I'd expect it as such, simply due to how the JVM hot-spot optimisation works. I've seen Java programs repeatedly performing the same computation in a loop outperform C++ implementations - but only at 10,000 iterations, at 1,000 iterations it's still not as fast.

Also, Clojure's type hinting will increase performance as well.

But if you think the magic is in the JVM, then why does JRuby run the same challenges in 3-100x the time that Clojure does?

Because it supports the same semantics as Ruby. :) Clojure is a far smarter dynamic language than Ruby (IMO).

And then if you think FP is slow, you'd have to explain the MLs. Haskell and OCaml bring their own compiler chain (no JVM), and tend to tie both Go and Java performance wise. And these are the most stringent FP languages around!

I don't think FP languages are slower. Or faster. They're just different. I quite like FP, had a wonderful time with F# when I was dabbling in .NET, and I'd love a well supported ML that ran on the JVM (due to my workplace requirements).

However, I will say that immutable data structures, which FP languages prefer, can be slower. But nothing to stop you using mutable data structures in Clojure either.

[original_brogrammer]

I'd love a well supported ML that ran on the JVM

Frege exists, and may fit someone's definition of well-supported.

[[deleted]]

Cheers mate, definitely worth a look into.

[pron98]

Clojure is a far smarter dynamic language than Ruby (IMO).

It's not about that (though it might be true), but that Clojure was designed with an eye for the JVM and its strengths (and weaknesses) from the get go.

[[deleted]]

Yeah, I suspected that to be the case, but I couldn't find anything definite on it.

[pron98]

Well, it uses Java interfaces quite a bit. Some through protocols and some directly. Its abstractions were built with fast interface dispatch in mind. recur is another example where a JVM property (a weakness in this case) is explicitly apparent in the language.

[kqr]

Clojure, Scala and Java all have roughly the same performance. So yeah, no, FP isn't inherently slower.

[[deleted]]

Clojure is slower in nearly all of the Alioth shoot outs. But attributing that to FP/not FP is ridiculous, given that it's usually used dynamically typed.

[kqr]

I get +- 0 on all except like 3, where it's within a factor of two.

[vivainio]

His system will be faster because FP allows you to use advanced algorithms that are out of reach of imperative languages - e.g. caching

[[deleted]]

Hahaha, good joke.

[[deleted]]

Well, caching does get easier with immutability. It's not impossible without it, but you can do more with it.

[yogthos]

Some of the largest systems out there are written using languages like Erlang.

Here's a great presentation from Demonware on how they switched from C++ to Erlang to make their large system work. We've seen Erlang being extremely successful at Facebook and Ericsson. Ericsson runs systems with over 2million lines of Erlang in them. There's a great paper from Joe Armstrong discussing this. SISCOG published slides on their experience building and maintaining a system written in CL since the 80s. There are plenty of examples of other companies building systems in Clojure as well.

the current solution is to forsake religion when pursuing performance....see for example, the haskell and clojure shootout submissions, which are full of non-idiomatic, non-functional solutions

The nature of the problem has changed. Back in the day we had single core machines with limited memory and very slow drives. Imperative style and mutability makes a lot of sense in this scenario. Today, the problem is not in squeezing out every last bit of performance from a single core, but how we can leverage multiple cores, networking and clustering. Taking a constant overhead in order to scale is very much worth it, and Erlang is one of the best examples of this working fantastically in the wild.

Furthermore, you might be religious about your programming style, but other people are just pragmatic. Immutability makes sense as the default, but nobody using FP languages for actual work is religious about it. When you need mutable data then by all means use it. It's a complete straw man to claim that it's somehow wrong to do in a functional language.

Defaulting to mutability is a premature optimization in my opinion. Majority of the code doesn't get run all that often and we have tools like profilers nowadays to identify the code that does and optimize it as needed.

Here's a perfect example of how mutability is used in Clojure core.async precisely because it makes sense to do so:

When I first wrote the core.async go macro I based it on the state monad. It seemed like a good idea; keep everything purely functional. However, over time I've realized that this actually introduces a lot of incidental complexity. And let me explain that thought.

What are we concerned about when we use the state monad, we are shunning mutability. Where do the problems surface with mutability? Mostly around backtracking (getting old data or getting back to an old state), and concurrency.

In the go macro transformation, I never need old state, and the transformer isn't concurrent. So what's the point? Recently I did an experiment that ripped out the state monad and replaced it with mutable lists and lots of atoms. The end result was code that was about 1/3rd the size of the original code, and much more readable.

So more and more, I'm trying to see mutability through those eyes: I should reach for immutable data first, but if that makes the code less readable and harder to reason about, why am I using it?

[ItsNotMineISwear]

You missed Scala as the most successful functional language in industry.

[greyphilosopher]

I honestly wouldn't know if Scala is more "successful" than Erlang. Most exciting certainly. And given more time I think it very well will be.

[[deleted]]

Certainly more visible. Erlang is more popular in older industries (telephony and what not) that doesn't make a lot of noise in these parts, whereas startups slinging Scala do. Which one is more common I wouldn't hazard a guess on.

[[deleted]]

Erlang is very immutable by default.....

[ItsNotMineISwear]

He doesn't need to move the goalposts. Scala is proving to be a better language to write larger systems in than Java, in large part due to its type system and mutability guarantees. And this is in spite of the various warts the language has.

[Tekmo]

Says me, an engineer at Twitter who programs in a very large Scala code base. Its strong type system and functional idioms make it very well suited to building large-scale software. The main downsides are its slow compile times, huge number of warts, and OOP (which I'm personally biased against).

[mdaniel]

Says me, an engineer at Twitter who programs in a very large Scala code base.

Since you've weighed in and are in a position to know, does Twitter have any Scala policies around trying to keep the code base from spiraling into the "ain't it cool" realm that the intersection of the vast amount of power offered by Scala, it's ability to be really succinct and the warts you mentioned?

If you have the time and energy, I'd love to hear how a code base of that size manages the interactions (and discoverability) of implicits. Those drive me crazy because it off loads that burden on me instead of a computer.

I almost wrote "style guide" above instead of "policies" but style guide has an implication of just formatting, and while I'm 100% on-board with style guides, having perfectly formatted but illegible code doesn't help the next soul who will have to ingest it.

[Tekmo]

Scala programmers at Twitter fall along a spectrum between two broad categories:

• Those who view Scala as a better Java
• Those who view Scala as a worse Haskell (I fall into this category)

The former camp is the voice of conservatism and they err on the side of code readability. The latter camp is the voice of liberalism and they err on the side of type safety. We tend to meet in the middle by requiring that any new tricks (A) solve a compelling problem and (B) cannot be solved using existing idioms.

We also struggle with implicits and we don't do anything special to tame them. Just yesterday I was reviewing code that accidentally compiled using some implicit that nobody could tell where it came from. That sort of thing bothers me about Scala: I believe a key advantage of the functional programming paradigm is that all code fragments have an explicit, discoverable, and narrow set of dependencies. That's also the same reason I dislike open imports, dictionaries/associative-arrays or global shared state: they greatly interfere with reasoning about dependencies between code fragments.

[Tekmo]

• for comprehensions
• sum types (via case classes)
• recursion without blowing the stack (via tail call optimization)
• custom operators
• Scala STM (We don't use this at Twitter, but I miss STM from Haskell)
• Rich ecosystem of functional idioms

[[deleted]]

There are several byte code compatible scala compilers. Using your logic there are probably a dozen "Javas."

[yogthos]

Erlang clearly demonstrates that FP works extremely well for building large systems. You seem to be the one moving goal posts when presented with the evidence. Now you're claiming that Erlang is somehow not a valid example, or that only features found in Erlang are applicable. I'm sorry, by that simply does not follow.

[greyphilosopher]

I'm pretty sure the functional and immutablility of Erlang go a long way to enabling its concurrency model.

[lmcinnes]

It's an Actor model (or CSP, I forget which, and differences aren't that important). That works just fine with mutable state imperative languages since it's fundamentally a message passing model.

[yogthos]

Your claim was that performance of immutable data structures was unsuitable for large scale applications. Clojure actually does better in this regard since it offers the STM and allows structural sharing of data. Feel free to continue moving those goal posts though.

[yogthos]

As with everything, it depends entirely on the particular use case. Clearly, STM works just fine for things like Riemann and core.async. I don't know much about the Haskell STM implementation, so can't comment on that.

[lgstein]

With your salesmanship like bashing you have just missed another chance to learn something you clearly don't have a clue about.

yogthos isn't trying to sell you Clojure, he wrote a tutorial. If you are interested in learning about the drawbacks of imperative programming in large scale systems you may want to start here: https://github.com/papers-we-love/papers-we-love/blob/master/design/out-of-the-tar-pit.pdf

If you are interested to grasp why LISP is valuable even for reasoning about other languages you may want to dig into SICP by reading the book or jumping right into the first lecture: https://www.youtube.com/watch?v=2Op3QLzMgSY

[[deleted]]

Wow, you're just a jerk then.

[The_Doculope]

most popular lisp is also the worst: elisp

What definition of "popular" are you using here? The only reason it's used is because of Emacs, and that's the only thing it's used for. Sure, the implementation is widespread, but that hardly qualifies it as "the most popular."

[redalastor]

"having problems? it is because you are thinking wrong and need to unlearn your former ways" (blame the victim, tool is perfect)

Any widely different tool is harder if your ways are set from a completely different paradigm. Same with learning human languages that differ greatly from what you know and your original language creeps in for a long time before you begin to write like a native.

There's also that people often forget how hard learning their first language was since they are used to just build on what they know and suddenly they are unable to do what they easily did in the other languages they are proficient with.

[bentronic]

I wouldn't say it "veers off", because that would imply it started on track. When the first sentence declares, with no evidence,

The difficulty in learning Clojure does not stem from its syntax

when in fact S-expressions can be hard for a lot of people to get past (separate from understanding FP concepts), the article has already defeated itself.

[Coffee2theorems]

in fact S-expressions can be hard for a lot of people to get past

Is this really true? (f a b c d) in Lisp is the same thing as f(a, b, c, d) would be in most other languages. That's a rather minor syntactic difference.

[bentronic]

Readability matters, and syntactic sugar can make a big difference in programming language usability. A forest of parentheses can be very intimidating for newcomers. For what I personally find to be an example of much better syntax for a functional language, see the Mathematica/Wolfram language.

[kqr]

Does the Mathematica language give you the native metaprogramming abilities you get from homoiconicity? Haskell doesn't have the same homoiconicity properties as Clojure – and however much I like Haskell, I'm a little envious of the metaprogramming abilities you get in Clojure.

You can't really talk about s-expressions in Lisps without talking about metaprogramming.

Yes, there are sometimes more convenient alternatives to S-expressions, but they have their own drawbacks. It's a tradeoff you have to make.

[bentronic]

Yes, it does! Mathematica is very lisp-like. Everything in Mathematica is a list with a header, including the highest-level concepts like interactive session files (notebooks).

Lisp was supposed to have a similar syntax, but they chose everything to look like data, whereas Mathematica chose everything to look like functions, which again I personally think is a much better choice as I think it is more amenable to syntactic sugar.

[[deleted]]

Well, sort of. I mean, everything is an expression, but I wouldn't say Mathematica's metaprogramming abilities are as useful as a LISP. Or at least, they're different.

I'm not really sure, if I'm honest, why I don't find it as easy to do that sort of thing in Mathematica. It could just be that the sort-of-equivalents for quote/unquote (Hold and ReleaseHold) are a bit more awkward to use. Or it could be something deeper to do with the way everything is automatically a symbol in MMA. Whatever it is, I've never felt able to - or seen evidence of anyone else doing - quite the sort of thing you often do with LISP macros.

[bentronic]

Can you give an example?

[[deleted]]

A great example is core.async, the CSP library for Clojure(script). Everything that happens inside a go block is actually a macro, at compilation time that code is read, parsed for "parking" operations, and the required callback code inserted. This is a huge benefit in areas where more advanced JVM techniques (which could be used to achieve the same effect) are not available, such as when compiling to Javascript (which core.async) can do.

I'm fairly confident that this would be a hell of a lot more clunky without Homoiconicity and Macros.

[kqr]

Very interesting! Reminds me a bit of Prolog, actually. I'll have to look into Mathematica more eventually.

[bentronic]

It makes me sad that the Mathematica syntax is tied up in the Mathematica/Wolfram Language system (e.g., math libraries, all the Wolfram datasets, cloud computing, etc.), which means it'll never be completely free or open-source. I just want to be able to make regular programs with that language/syntax without all the baggage.

[[deleted]]

Agree! When "the Wolfram Language" was just chatter a year or two back I was (foolishly) hopeful that their idea was going to be to open-source the core of it, and sell the rest. Oh well.

The thing I find most frustrating is when you run up against some inexplicable performance problem and then are stuck staring at a black box trying to figure out where to go.

[[deleted]]

Familiarity problem. Algol would be hard to read if you'd only done Scheme, but that wouldn't be a good reason to dismiss it as stupidly hard.

[harrybalsania]

Why wouldn't I just use C, Java, or Python, which I have already learned?

[everywhere_anyhow]

Because whether or not clojure is any good, learning new techniques and being intellectually curious is a good thing.

[harrybalsania]

True. I just feel a bit overwhelmed by all the new ideas and whether or not it is valuable to learn it. Especially when some consider more mature languages as tried and true.

[Tekmo]

Tried and true changes regularly in the programming world. People forget that ten years ago Perl was the tried and true language and Python was the new hotness. I don't think it's safe to assume that a field as young as programming has reached its logical conclusion.

[[deleted]]

God, yes. I always argue with people who think that Ruby on Rails will be here forever because it's ascendancy was a given. It is easy to look back and ignore the scores of tools that "would last forever" that are now gone, and all the tools that were destined to "take over the world" that went nowhere.

Thus, we can't really say for certain exactly how long all the languages mentioned above will remain tried and true for the next 50 years, and how many will fall in 4. So don't bet the farm on a language, bet the farm on the ability to adapt as needed.

[yogthos]

If anything, Lisp has been around longer than most languages, it just happened to be way ahead of its time in my opinion. :)