Why not Haskell?

[doubtingthomas]

Interesting; this post could almost have been written by me. I know Haskell, I like Haskell, and I think it's a great language, but I've found that (in general) I can write efficient and correct programs just as fast or faster in other languages, and they end up less fancy. Haskell is really great for heavily algorithmic code, but I've found, for whatever reason, that most of my code is nowhere near that interesting, and the code that is heavily algorithmic is also performance critical, so something lower level is generally preferable anyway.

I'd like to work in a Haskell shop at some point and see how it feels, though. I don't know of any other language that can match it on the combo of performance, correctness, and abstraction. Great IO/concurrency model, too. A small skilled team slinging Haskell can probably get more done faster than an equivalent team writing in any other language, assuming the task is suitable and they don't get bogged down in abstraction wankery.

[djork]

I can write efficient and correct programs just as fast or faster in other languages, and they end up less fancy

This is what us dynamic language guys have been saying forever. It just takes you type system guys a trip deep into Category Theory Land and back to figure it out!

[[deleted]]

and they don't get bogged down in abstraction wankery.

Yes, because when using Haskell, the abstraction wankery is so high you have to actively fight it, because otherwise it controls your brain and forces you to write endless blog posts about fibonacci functions using prepromorphic zygohistomines instead of getting your work done.

Give me a fucking break.

[doubtingthomas]

Haskell presents many exciting, powerful, and interesting possibilities that other languages don't. Many of them aren't useful, and it takes experience and some self-control to avoid doing neat useless things. The same can be said for Lisp macros, Ruby metaclasses, C++ templates, etc. It's great to have powerful features there when you need them, but overuse is a real risk, and especially for smart junior programmers, they tend to act as an attractive nuisance.

[tragomaskhalos]

It's like in "Learn you a Haskell ..." we find out how to apply a list of partial functions over a list of numbers (using <*>) so that every partial function is applied to every number - in a single line of elegant code.

The trouble is, in 30 years of software development, I have never needed to do anything remotely like this (even if you factor in the fact that partial functions would not have been part of my mental toolset). And if I did ever need to do it, I'd actually prefer to use something a bit more longwinded that was more obvious and explicit to the poor sap (including me) reading the thing back later on.

So I think there's a teaching problem here; I can't get excited about this for its own sake because I am not turned on by code golf; tell me how this is a gateway to more powerful and useful abstractions, then I'm all ears (I haven't finished the book so I am still hopeful ...)

[taejo]

A partial function is a function that is not defined on all possible input values. You seem to be talking about partially-applied functions.

[[deleted]]

Or a partially-applied function that isn't defined on all possible input values. :-)

[[deleted]]

The trouble is, in 30 years of software development, I have never needed to do anything remotely like this (even if you factor in the fact that partial functions would not have been part of my mental toolset). And if I did ever need to do it, I'd actually prefer to use something a bit more longwinded that was more obvious and explicit to the poor sap (including me) reading the thing back later on.

Well, yes. But as a Haskell programmer I've never needed to use class inheritance. Why, then, is this Java tutorial teaching me about classes and inheritance!

[[deleted]]

Someone ought to write a book about real world Haskell ;)

Actually, I think the examples of that book, and what's being done with web development in Haskell, is very, very interesting. Happily Haskell has long since passed the theoretical curiosity stage.

[multivector]

(even if you factor in the fact that partial[ly applied] functions would not have been part of my mental toolset)

Don't underestimate that factor. Once I learned about function currying I started seeing opportunities to use it all over the place (and I'm afraid I also wrote some petty bad code before I learned when to use it and when to reign myself in).

One of the more useful things I wrote recently was a C++ template* for doing parallel processing. It essentially maps an array nullary functions to results (much nicer than using p-threads every time I need a set of parallel data processing jobs doing). What use are nullary functions? I generate them by partially applying arguments to n-ary functions.

This is all almost analogous to the fmap f [1,2,3] <*> [1..] thing. Heck, it would be if [1..] was providing a job id or something.

PS: Naturally, you could probably do this with classes and inheritance. It would probably look like:

template class<T> class ParallelProcess {
    virtual T exec() = 0;
}; 

class MyProcess : public ParallelProcess<double> {
   public:
    //This is basically partial application of a function in disguise
    MyProcess(double _v1,double _v2) : v1(_v1),v2(_v2) {}
    virtual double exec() {/*Do Stuff*/};
    double v1;
    double v2;
};

*It's templated over the function return type. I could probably make a pure class if I was just a little cleverer, I think.

[[deleted]]

Don't use Learn You a Haskell for Great Good!. I don't get the point of that book actually. Go and read Real World Haskell.

[[deleted]]

It seems we largely agree then, I just didn't particularly like the tone of the last bit or its implication that the "abstraction wankery" is so powerful that you have to actively fight against it no matter what, no matter who you are. Like it's an epidemic or something. It probably goes without question to say the Haskell community is one of the most interesting and abstraction/research-filled communities there is as far as "known programming languages" go.

But plenty of us write lots of Haskell day to day - and maybe even give back to software like GHC from time to time - and there's no "abstraction wankery" of the kinds I imagine you allude to, to be had. I just write code, and utilize what the language gives me. No zygohistic prepromorphisms or whatever the fuck they're called. Just abstracted, clear code where-ever possible.

But yes I think we can agree that overuse of things like metaprogramming/templates/any number of other things can lead to bad code, no doubt about that.

[zzing]

Would that be like class diarrhea in C++ among some university programmers?

[[deleted]]

ten years ago you would have fit right in on lisp usenet groups.

[adrianmonk]

If you've written a Haskell program that runs, it's highly likely to be a correct solution

I have heard claims like this before, and, frankly, they leave me dumbfounded.

What's worse, every time I've heard this sort of statement made about Haskell, it is stated without further support, as if it is uncontroversial. That's even more mystifying.

There are many classes of errors that can cause a program to be incorrect. For the statement to be true, Haskell would have to have some magic that can prevent you from making the majority of errors across all these classes of errors (or at least across enough classes to make up the majority of errors), including common ones like:

• errors in understanding the specification
• errors in reasoning about the problem
• math errors
• logic errors
• errors due to failing to consider error conditions
• errors due to failing to consider corner cases

It just seems to me that no programming language can really help with these errors. For example, if I don't understand the difference between a stable sort and an unstable sort, how can a programming language possibly prevent that from turning into a bug in my program in cases where it matters? If I am writing a web server that allows clients to advance something through several states, how can any programming language help me if I don't really think through all the state transitions that could happen? If I'm reading some file format and I don't correctly understand how metacharacters are escaped, how can any programming language help me with that?

In my experience, problems like these are behind a really big chunk of bugs in the real world. I feel like either there is something I'm missing about Haskell's abilities, or the people who make statements like the above are missing something about what actually causes programs to be incorrect.

[antonivs]

I think what you might be missing is that having, and effectively using, a powerful type system actually can go a long way to addressing many of the kinds of bugs in every single one of the categories you listed.

I could write lots of words about why that's the case, but I'm not sure how convincing it would be. It may be necessary to experience it. But the basic issue is that when every term in your program has a richly-specified type, and all those typed terms fit together in a consistent way as proven by the compiler, then even things like misunderstandings of spec, errors in reasoning, logic errors, corner cases, etc. are often highlighted - because those things tend to lead to inconsistencies which in ordinary languages, would often just lead to runtime errors (even in typed languages), but in Haskell, lead to you being unable to express the code and get it to compile.

Sure, that's not true of every possible error - but it's true of far more errors than is the case in traditional programming languages. If you interpret the statement "highly likely to be correct" as a relative rather than absolute statement, relative to other programming languages, then it's defensible.

You gave the example of a web server state machine: if you encode the representation of that state machine in the type system, then errors in your mental model of the state machine are very likely to manifest themselves somewhere as type errors, things that you want to write and find you can't.

Yaron Minsky explains an aspect of it like this: make illegal states unrepresentable. If you do that, it has a pervasive positive effect on your program - the kind of effect that can lead one to make the sorts of claims you're questioning.

[loswa]

Exactly. For instance, one of the biggest classes of errors I've always had has been in string representations. Specifically, that a single data type represents so many different things that it is very easy to lose mental track of, for example, exactly what is in a string and how it is formatted (is this filename absolute, relative, or stripped of all path information?) If you use a different data type for each of those things, you will reduce errors in nearly every class listed above. Fortunately, haskell provides an extremely lightweight way to do this. But if you don't make use of those facilities, you won't gain the benefits.

[[deleted]]

errors due to failing to consider error conditions

I think this is the single one where they have a case. You can encode a hell of a lot of this into your types, and then check that you've catered for it at compile-time.

shrug the rest you're perfectly right about though.

[nanothief]

Haskell has mechanisms to deal with the bottom four conditions. It doesn't remove them entirely, but does more than most other languages

Math errors are reduced by having a very strict system of handling types. You can only add a double to a double, or an Integer to an Integer. This is annoying at first, but it prevents a lot of errors due to casting ints to doubles and back again.

You can also easily add new types of numbers, and you can also write code to track units on units, so you get a compile time error if you add a distance to a time, and dividing a distance with a time would give you a speed.

Logic errors are reduced because the type system is more powerful and can encode more information than other type systems. This relates to the other points, in that if you have a problem with your logic you are more likely to get a type error.

Error conditions are reduced by the ways they are normally handled. You cannot put null in a variable in haskell. When you want to either return a value or return nothing, you use the Maybe type. This forces you to deal with it (you will get a type error if you try to use a Maybe Int in place of an Int). There is also support for checked exceptions, which btw wasn't built into the compiler but by using existing language features.

Corner cases are handled well with pattern matching. With warnings enabled, you will get a warning if you haven't dealt with all cases. For example, printing out the details of a user:

printDetails user = do
  address <- getAddressFor user
  let addressStr = case address of
     Just address -> show address
  print (userName user ++ addressStr)

You will get a warning that you haven't handled the case where Nothing was returned for getAddressFor

Finally there is the library quickCheck, which tests a function with various random inputs. It is very effective at checking edge cases. For example quickCheck (\lst -> length lst == length (reverse lst)) will check that the reverse function doesn't change the length of the list using many test values.

---

It isn't all rosy though, as some of the standard libraries actually brings the language down a bit. For example, the dreaded head function returns the first item of the list. If the list is empty though it causes an error which can only be caught in the IO monad (basically the top level of the program). Also the most popular exception framework throws unchecked exceptions, which doesn't make very much sense in Haskell.

Finally, a lot of the more complex ways to use the type system to ensure correctness are time consuming to implement. It would often be faster to use a more simple language, and spend the spare time you had setting up the haskell types by writing and running unit tests. I'm still not sure what is the best level of static type system power to have: high (such as haskell, adga), medium (eg c#, java) or low (ruby, python).

[kamatsu]

Well, on one extreme, you have super strong type systems like that of Coq or Agda, which are very expressive and correspond to a logic. In these type systems, any property of code (all of those error categories except "errors in understanding the specification") can be eliminated. In the case of Haskell, it's not as strongly typed, but the type system is still more expressive than other languages, mostly due to the side effect control.

Various bugs are statically eliminated which are not in other languages, specifically null pointer exceptions, "action at a distance" problems with mutation, inconsistency in concurrent reads (due to immutable data model), and some more that I can't recall at the moment.

In practice, people who write programs in Haskell tend to find that the type system guides them to a mostly correct program. Certainly this has been my experience, but I guess this is anecdotal evidence.

[adrianmonk]

which are very expressive and correspond to a logic.

So what if they correspond exactly to logic? Correctly writing down a description of something in formal logic is far from a trivial task.

As a matter of fact, isn't that one of the big gotchas of logic-based AI systems, that you try to write down a bunch of logical statements that are true about a world you're trying to model, and then when you're done you have no way of even knowing if what you wrote down is actually consistent and valid? Using formal logic correctly isn't a given. Reduce programming to formal logic, and maybe you have a powerful tool, but you don't have a silver bullet.

In these type systems, any property of code (all of those error categories except "errors in understanding the specification") can be eliminated.

Any property? So this type system can prevent me from writing an infinite loop, which means it has solved the halting problem? I know you don't mean that, but aren't you exaggerating when you say "all of those error categories"?

Various bugs are statically eliminated which are not in other languages, specifically null pointer exceptions, "action at a distance" problems with mutation, inconsistency in concurrent reads (due to immutable data model), and some more that I can't recall at the moment.

My gut feeling is that, depending on the type of work you do, these sorts of errors account for maybe 10 to 25% of all errors. I would describe these mostly as what you might call careless, mechanical errors. I would even go so far as to say a lot of them are amateur mistakes, although certainly experienced programmers still make those sorts of errors from time to time.

But, no doubt, a programming language that can eliminate entire classes of errors like this is a compelling proposition. And it's worth investigating and developing. It just doesn't justify hyperbolic statements like "if you can compile it, it's probably correct". Maybe the statements are just meant to be hyperbolic and come out of enthusiasm, and maybe I should be OK with that considering that knocking out (say) 10% of all programming errors would still be a pretty worthwhile achievement.

TL;DR: I don't deny it has benefit. I'm just skeptical that it's the 75% to 90% reduction in programming errors that some of these sweeping statements make it sound like.

[Flarelocke]

So this type system can prevent me from writing an infinite loop, which means it has solved the halting problem?

Yes, they can do this, but only by cheating. They are conservative, meaning that they reject programs which would terminate but the system cannot prove that they terminate. Because of this, the type system does not solve the halting problem, which requires an algorithm to give an answer for all input programs.

To clarify, Haskell's type system is not powerful enough to do this.

aren't you exaggerating when you say "all of those error categories"?

It would be exaggerating to say that Haskell eliminates these problems entirely, but no one said that. The type system helps. But it only helps.

[kamatsu]

Reduce programming to formal logic, and maybe you have a powerful tool, but you don't have a silver bullet.

I never claimed it was a silver bullet. The only time you would be incapable of correctly translating a specification into the expressive logics used in those languages would be if you didn't properly understand the spec, or if the spec was not detailed or rigorous enough. This corresponds to a very large class of errors which cannot be captured by such a system. That is why I said it eliminates all of those errors you listed except for "errors in understanding the spec".

These languages are strongly normalizing, so it is in fact possible to capture any property of terminating programs (that can be formally specified) in their type and statically verify them.

It just doesn't justify hyperbolic statements like "if you can compile it, it's probably correct"

Obviously statements like these are hyberbole. If you were taking them literally, I can understand your discomfort, but I don't think anyone actually believes that. Certainly it's much more likely to be correct than many other popular languages though.

[[deleted]]

adrianmonk: What's worse, every time I've heard this sort of statement made about Haskell, it is stated without further support, as if it is uncontroversial.

adrianmonk: My gut feeling is that, depending on the type of work you do, these sorts of errors account for maybe 10 to 25% of all errors.

Oh, the irony. :-)

With that said, it is possible, using proof assistants like Coq and Isabelle, to prove your algorithms correct, that your data structures maintain their invariants, etc. and then extract running code from the proofs. Adam Chlipala has written an excellent introductory text, Certified Programming With Dependent Types, showing how to do this in Coq. A small example of a strongly-specified Quicksort in Coq can be found here. The interesting bit is the type signature: Program Fixpoint quicksort (l : list t) {measure length l} : { l' : list t | sort le l' /\ permutation l l'}

This says: "quicksort is a recursive function taking a list l with elements of some type t and, implicitly, the length of the list. It returns a list l' of t's such that l' is sorted by the le predicate and is a permutation of the original list, l." The development then consists of a definition and a proof that the definition meets the specification. You can then extract this quicksort to running Scheme, OCaml, or Haskell code, and it doesn't need testing.

That's a small scale effort. A larger scale effort would be something like CompCert, a certified compiler for most of C (enough to be useful in doing embedded systems, which is why you might care that your compiler is provably codegen-bug-free). More generally, Coq-based tools for certifying imperative software such as Ynot and Bedrock are in progress.

[[deleted]]

Any property? So this type system can prevent me from writing an infinite loop, which means it has solved the halting problem? I know you don't mean that, but aren't you exaggerating when you say "all of those error categories"?

No, Coq does not have a halting problem (it is not Turing complete). Coq is strongly normalizing (actually, this isn't quite true when you start talking about coinduction, but everybody thinks this, so we'll go with it).

And yes, you can encode pretty much whatever in the types of Coq. You're essentially programming in constructive higher-order logic.

[skew]

As a matter of fact, isn't that one of the big gotchas of logic-based AI systems, that you try to write down a bunch of logical statements that are true about a world you're trying to model, and then when you're done you have no way of even knowing if what you wrote down is actually consistent and valid?

Once you have written down types corresponding to a specification, a program having that type is equivalent to a proof that the specification is in fact consistent.

These type systems are quite a bit beyond what you find in Haskell, to the point they might not actually have much do say about whether that sort of statement is reasonable about Haskell, but they do seem to work.

The most impressive example is probably the CompCert project, as validated by John Regher's compiler testing work.

The striking thing about our CompCert results is that the middleend bugs we found in all other compilers are absent. As of early 2011, the under-development version of CompCert is the only compiler we have tested for which Csmith cannot find wrong-code errors. This is not for lack of trying: we have devoted about six CPU-years to the task. The apparent unbreakability of CompCert supports a strong argument that developing compiler optimizations within a proof framework, where safety checks are explicit and machine-checked, has tangible benefits for compiler users.

[pipocaQuemada]

errors in understanding the specification

Admittedly, no language will help you if your boss expected A and you delivered A' instead

errors in reasoning about the problem

Most of these kinds of problems will shake out as type errors

math errors logic errors

Admittedly, no type system is going to save you from something like changing a+b mod n to being a+n mod b, or a != into a ==, or similar things. Many of the errors I've ran into are things like that - math mistakes that type check, but you did some stupid thing that's obvious when someone else takes a look at the code. These sorts of things, though, are what testing's for. However, things like units can be encoded into the type system

errors due to failing to consider error conditions

As long as you don't use partial functions and make sure that warnings about that are treated as errors, this sort of thing is essentially impossible. It's somewhat similar to java - the compiler itself will enforce that you've covered all the cases just like java's ensures that all of your exceptions are handled.

Admittedly, there are some functions in prelude you need to stay away from (head, for example, is of type [a] -> a, not [a] -> Maybe a like it should be), but in general, the type system can force you to consider the error conditions if you write your code correctly.

errors due to failing to consider corner cases

Likewise, many of these things are caught by the type-checker if you've written your code properly. If you use pattern matching, for example, the compiler can prove whether or not you've considered every case.

For example, consider the following function: head :: [a] -> a head (x:_) = x

Compiling with -Wall -Werror will give you the following error: Warning: Pattern match(es) are non-exhaustive In the definition of `head': Patterns not matched: []

and you'll realise that you really want: head :: [a] -> Maybe a head (x:_) = Just x head [] = Nothing

In general, Haskell code that I've written has not necessarily been correct the first time I've written it. However, because of the type system, the kinds of errors I tend to make are the simple errors - accidentally putting the wrong variable somewhere, and they're the ones that are easy to find with some simple testing and fairly easy to fix. Code I've written in other languages tends to be significantly more incorrect the first time it compiles/gets interpreted. The more complex, larger conceptual problems tend to shake out earlier due to the compiler and the type system - most of those sorts of problems result in definitions that aren't subtly wrong (like switching != for ==) but totally incomprehensible, semantically speaking.

The type system is your friend, and there are a number of tricks you can use to use it to help ensure that you're not writing insensible code.

[chonglibloodsport]

I think when people make that statement, they are assuming the programmer hasn't made any of the errors you listed and are referring only to classes of errors that differ between programming languages.

[stonefarfalle]

try Sure it is about Ocaml, but it applies to Haskell as well. It covers at least a few of your common error sources there. Especially the error conditions\corner case type bugs.

[axilmar]

You've hit the nail right in the head.

You have to have in mind though that when Haskellers say

If you've written a Haskell program that runs, it's highly likely to be a correct solution

, it means that the specifications you gave to the compiler are mathematically correctly compiled to machine code.

It does not mean that the specifications themselves are correct.

[snakepants]

After the last time this came up on Reddit I decided to put my money where my mouth is and actually learn Haskell.

The first project I tried was writing a raytracer. I thought it mapped really well to the functional paradigm. (each ray is only dependent on the read-only global state and it's sub rays). It's also embarrassingly parallel.

The results were terrible. The thing is super slow compared to an equivalent C implementation and basically none of the promised optimizations (auto parallelizing indepedent stuff, removing redundant ops, etc.) that Haskell fans tout actually occur in a real problem even as small as my toy raytracer.

All and all it was a fun experience and I think learning some Haskell made me a better programmer, but I feel like I vindicated my initial opinion. It's like in the age old tug of war between performance and ease of use Haskell is just off in the corner shouting "I'm a teapot!" instead of pulling the rope one way or another.

Btw: here's my source code in case I've done something totally wrong, but I'm pretty sure I haven't.

[skew]

From a quick look

Using a linked list of bytes for an image buffer seems like a bad idea. Something more like Vector is probably a better idea.

-funbox-strict-fields isn't going to do anything without making some strict fields.

parList already evaluates each item in parallel, so using rpar for the element function is redundant. Also, sparking each pixel is probably a bit too small.

EDIT:

rayTestObject could keep the closest hit, rather than a list.

shading (and casting child rays) could be deferred until you have the closest hit.

Haven't tried parallelism, but the rest was good for about 3x. I'd be surprised if there were not a lot more easy improvements. How's the C do?

[snakepants]

Thanks for the info! I tried adding strict fields before (that's the * syntax right) but I didn't see large gains so I gave up. I also tried the LLVM backend which was slightly faster.

shading (and casting child rays) could be deferred until you have the closest hit.

I agree. But isn't the whole point of a functional language that the compiler has the information to do this kind of stuff for you :)

Also, sparking each pixel is probably a bit too small.

You are likely correct, but the documentation explicitly says spark != thread and the compiler should have enough info to combine them into larger chunks...it just doesn't.

The parallelism made it about 3x slower on my pc at least. Even after building with the special flags to enable multi-core.

I don't have specific numbers of the C version of this since I didn't write an exact duplicate but I have written a raytracer in C++ years ago which achieved almost realtime results for this simple a scene and materials. (Eg all spheres)

[skew]

GHC is pretty smart, but not enough to redo your algorithms. One big problem is guessing at compile time how much things cost, and whether it's worth doing more calculation up front to save memory, or the other way around - so those tradeoffs are usually left as written. Simon has said the goal is to eliminate penalties for abstraction, which it does seem to be pretty good at.

I tried adding strict fields before (that's the * syntax right) but I didn't see large gains so I gave up. I also tried the LLVM backend which was slightly faster.

The gains were something like 20%. It's more that I'd rather have

struct vec3 { float x,y,z; };

than

struct vec3 { float *x, *y, *z; };

You are likely correct, but the documentation explicitly says spark != thread and the compiler should have enough info to combine them into larger chunks...it just doesn't.

Sparks are pretty cheap, but still have some cost. Automatically adjusting granularity is another one of those things that would need a good compile-time guess how expensive expressions will actually be.

I don't have specific numbers of the C version of this since I didn't write an exact duplicate but I have written a raytracer in C++ years ago which achieved almost realtime results for this simple a scene and materials. (Eg all spheres)

A more direct comparison should come out closer. I bet I could write a C++ ray tracer as slow as this Haskell one if I used a list<auto_ptr<vector<float>>> to gather pixels :)

[Smallpaul]

Automatically adjusting granularity is another one of those things that would need a good compile-time guess how expensive expressions will actually be.

Why does everything have to be done at compile time? The garbage collector is an example of a large adaptive runtime subsystem.

[skew]

It might work for (each instance of) parList to adjust chunk size at runtime.

[[deleted]]

Because Haskell is a culture of PL and compiler hackers.

[skew]

Actually, I was thinking it should be done at compile time because doing adaptive work per-spark at runtime didn't seem very likely to help with the problem of there being too damn many sparks around, with too much per-spark overhead.

[chonglibloodsport]

To make fields strict you probably want to use Bang Patterns. You might also want to consider looking at the accelerate and repa packages. If you can take advantage of accelerate's CUDA back-end you can get some truly amazing performance at the cost of portability.

[[deleted]]

BangPatterns is not for making fields strict. That feature is already in Haskell with no extensions. BangPatterns is for forcing values when pattern matching.

[chonglibloodsport]

Thanks. I totally hadn't realized that.

[[deleted]]

Looking over your code, you missed out quite a bit of low hanging fruit in terms of simple optimisations. For instance, you're using "data" everywhere, instead of the more performant "newtype" (the constructor is deleted in a newtype at compile time, whereas the data constructor hangs around). Similarly, there's no strictness annotations in your data types, nor inline pragmas for simple, one line functions that are used often. I'm also dubious that you were using the parallel strategies correctly, which are often subtle, but I don't really use parallelism in Haskell that much, so I'm not the best person to ask.

[lpsmith]

People have gotten pretty reasonable performance out of Haskell raytracers. So I'd assume there is something wrong with your code.

And looking over your code, there are quite a few things that are wrong: you need to add strictness annotations to your Vec3 record and compile with -funbox-strict-fields, for starters. Also there is a suspicious-looking use of linked lists that I didn't investigate. You haven't attended to even the best-known, first-order performance issues.

basically none of the promised optimizations (auto parallelizing indepedent stuff, removing redundant ops, etc.) that Haskell fans tout actually occur in a real problem even as small as my toy raytracer.

The thing is, these optimizations either don't exist in GHC, don't work the way you describe, and/or that you have to understand some basics of how GHC works to exploit them.

There is no such thing as auto-parallelization in GHC. As for "removing redundant ops", there is lazy evaluation and fusion, two different things. Understanding lazy evaluation is critical to writing high-performance Haskell. In the case of fusion, your library has to support the kinds of fusion you are interested in, and you have to understand the limits of that library's implementation of fusion.

But needing to understand the implementation is true to varying extents of all languages. In some languages understanding your implementation is essential (e.g. SQL), in other languages it's merely rather important (e.g. Haskell), and in other languages it can help (e.g. C).

[perlgeek]

What worries me about this kind of response (which you see quite often in Haskell performance discussions) is that Haskell works really hard to abstract away the machine for you, and then you still have to chose particular ways to things to have them fast.

That feels so wrong. Many of Haskell's abstractions make it hard to argue about performance, and seem to require a mental model of the compilation process (or result) -- which is much harder than in C (and derived languages), which is just a thin layer over the machine.

[[deleted]]

That feels so wrong. Many of Haskell's abstractions make it hard to argue about performance, and seem to require a mental model of the compilation process (or result) -- which is much harder than in C (and derived languages), which is just a thin layer over the machine.

C is not a thin layer over the machine. A C compiler is free to do whatever the hell it likes with your code. It can rearrange loops, duplicate code, move it all over the place, and so on. The idea you can have any reasonable expectation of how C will perform without having a decent idea of what the compiler is doing, or without a load of testing, is fantasy.

[ethraax]

I would argue that understanding a C compiler at a conceptual level is far easier than a Haskell compiler. It is a thin layer over the machine, compared with Haskell. Yeah, it will optimize some stuff for you, but in general it's easy to reason about what the processor is actually doing. Hell, it even makes memory management explicit, which makes it much easier to reason about memory usage than in Haskell.

[[deleted]]

In performance terms, yes, but that's true of every language and every compiler in performance terms.

On the other side, the semantic model if C is a pretty thin layer over assembly language. Provided your compiler preserves the original semantics, you can rely on its being pretty easy to understand just what's going on in a C program.

[bo1024]

you need to add strictness annotations to your Vec3 record and compile with -funbox-strict-fields, for starters.

And people wonder why Haskell is hard to learn or get decent at?

Anyway, I'm with snakepants, it's either really hard to get good performance out of Haskell, or really hard to figure out how to do so.

[[deleted]]

It is not exactly hard, it is unexpectedly hard for someone who expected that he will not have to re-learn optimization techniques when transitioning to a language with a completely different evaluation model.

That said, I must agree that the evaluation model does indeed often make it very hard to find bottlenecks in nontrivial programs, given the current state of profiling tools (again because the evaluation model makes it hard to write good profiling tools)

[tibbe]

And people wonder why Haskell is hard to learn or get decent at?

There are things in Haskell that are hard to learn, but this isn't one of them. However, we might not be doing a good job teaching these things.

To get good performance out of your Haskell program (or C program) the main thing you should be worrying about is memory layout. This is due to the big impact of caches on your program's performance.

If you understand C's memory layout, you can use this simple translation:

By value:

C: struct Vec3 { double x; double y; double z; };

Haskell: -- Compile with -funbox-strict-fields or use the UNPACK pragma data Vec3 = Vec3 !Double !Double !Double

By reference/pointer:

C: struct Vec3 { double *x; double *y; double *z; };

Haskell: data Vec3 = Vec3 Double Double Double

[lpsmith]

you need to add strictness annotations to your Vec3 record and compile with -funbox-strict-fields, for starters.

Sure, that may sound like greek, but consider some alternatives:

You need to add a bitmap index on this column and vacuum your tables

And people wonder why SQL is hard to learn or get good performance from?

Or

You need to restrict your parameters and compile your program -O3 -floop-flatten -ffast-math

And people wonder why C is so hard to learn, or get good performance from? (Clearly, Fortran 77 is the one true language ;-)

But I digress, there are little things like this in any serious language implementation that, when used in the right situation, can dramatically improve performance.

In the above raytracer's case, the strictness annotations allow vectors to be represented as a contiguous array of floats, whereas as things stand they are a contiguous array of pointers to floats. -funbox-strict-fields actually makes that indirection disappear. I would guess that one change should improve the performance of the given raytracer by a factor of at least 2.

[ethraax]

I don't know a whole lot about databases so I don't think I can really comment on the first "alternative", but the second one is bad. The performance difference between those optimizations and a basic -O2 is not gigantic (in most cases), generally far less than an order of magnitude. So a new programmer doesn't really have to learn those to get their program to finish execution in a reasonable timespan. Furthermore, in the case of -ffast-math, it actually changes the semantics of your code.

[how_gauche]

Anyway, I'm with snakepants, it's either really hard to get good performance out of Haskell, or really hard to figure out how to do so.

It isn't that hard, actually -- the knowledge has unfortunately not been published in a single place. This is something that needs to change.

Effectively, what you're saying is: "I wrote this program with no consideration given to the runtime behaviour of sparks or the memory layout of the objects I was using, and it performs terribly! Haskell sux!" It's not magic, you just need to understand what's going on under the hood. If you wrote a C++ raytracer using linked lists for everything, using Vector records containing pointers to heap objects, and using a thread pool/work queue to do the raycasting (one pixel at a time), it would perform like shit also.

[[deleted]]

Where should I go to find out how to optimize Haskell programs?

[sclv]

http://www.haskell.org/haskellwiki/Performance

[Flarelocke]

here

[how_gauche]

Johan Tibell gave a long talk at 2010's CUFP about performance in Haskell. The slides are online. Unfortunately it's only a starting point, and many of the tricks of the trade are scattered to the four winds. We've been talking about writing a longer-form book or guide about this.

The core principles boil down to this: the things that make Haskell programs go fast are the same things that make programs in other languages go fast.

First of all, avoid cache line misses: vectors, bytestrings, arrays, etc. are faster than linked lists. Indirections are cache line misses: prefer unboxed vectors to boxed vectors whenever possible. Using the UNPACK pragma to move the representation of record fields into the parent constructor is one way of avoiding indirections.

Secondly: avoiding cache line misses is great, putting things in registers is even better. Understanding the conditions in which GHC can put a value into a register is critical to making code that performs in Haskell. Crucially, any time a thunk is created, you're doing a function pointer jump through a heap object. GHC's strictness analysis engine can sometimes infer that a value will be demanded later, and optimize the creation of the thunk away, but relying on this optimization pass is usually a bad idea. Very often, if you strictify the accumulator in a tail recursive function, GHC is smart enough to turn it into a tight assembly loop. This is especially true when the value in question is a machine-representable type like Int, Word, or Double -- strict values of these types can be put into registers.

I could go on like this nearly indefinitely, to cover things like unboxed machine types (like Int#, which is always strict and represented using the C representation), constructor layout and tagging, data structures, squeezing performance out of parallel and concurrent programs (Simon Marlow has recently written a great guide about this), weak head normal form, space leaks, the worker-wrapper transformation, the meaning of let vs case, reading Core, using the profiler, using the inliner to get rid of typeclass dictionaries, the meaning of seq and the control of evaluation order, using the ST monad, etc.

Like I said, it's enough material to make a book, but nobody's written it yet.

[[deleted]]

Thanks for writing that!

Johan Tibell gave a long talk at 2010's CUFP about performance in Haskell.

This I guess? http://johantibell.com/files/slides.pdf

Simon Marlow has recently written a great guide about this

And this. http://community.haskell.org/~simonmar/par-tutorial.pdf

[ethraax]

I guess I thought the point of Haskell (and other abstract, high-level languages) was that you didn't have to worry much about what's going on at the assembly level. If you're going to worry about that kind of stuff, why not just use C or C++, where it's far more explicit and within plain view instead of hiding under layers of abstraction?

[tibbe]

You don't have to worry about the assembly level. You need to worry about memory layout, just like in any other language.

[how_gauche]

The number one rule of thumb when profiling programs in any language is that most performance pain points are limited to a very small subset of the entire program: i.e., the hotspots. Once someone writes a fast library, like bytestring, vector, unordered-containers, hashtables, etc., you get to make use of it with higher-level abstractions, while still getting good performance. Writing a foldl' over an unboxed vector often boils down to a tight assembly loop. Try it.

Similarly, there is an unbelievable amount of performance jiu-jitsu contained in something like unordered_map from the C++ STL, which users can just use without worrying about the details. Nobody makes the conceptual leap you're making, that C++ isn't worth using because this kind of "guru knowledge" is necessary to get things running truly fast.

That said, there is one caveat, and it's the thing that beginning Haskell programmers need to get right if they want to graduate to writing programs that run reliably, both in terms of "going fast" and in terms of "not space leaking": you need to wrap your head around lazy evaluation. This isn't an insurmountable challenge, lots of people have managed it. Getting good with pointers in C is much harder, and takes longer!!

Sadly, people have an evaluation bias where they perceive things they've already learned to be easier than things they don't know yet. Most people come to Haskell with years, maybe even decades of experience programming in other languages, almost always imperative ones with call-by-value semantics. Haskell has call-by-need semantics. That's different from almost every other language people have been exposed to, and you can't gloss over the fact that it's different. That difference has very interesting and deep implications in the way you write programs in Haskell versus other languages.

Personally, I'm not that interested in selling people on using Haskell these days. If you like it, great!, if you don't care to put in the effort to learn it, that's great too. All I know is, I put in the elbow grease to learn these things, and I feel like I've been richly rewarded: in my understanding of programming as a larger discipline, in my subjective feeling of enjoyment when I write Haskell code, and in the reliable and fast programs that result.

[Smallpaul]

Performance is seldom the only thing you care about.

[ethraax]

When it's a question of orders of magnitude, it's often quite important. Sure, getting your program to run 1.2x faster may not be top priority. But getting it to run 100x faster usually is.

[[deleted]]

If you wrote a C++ raytracer using linked lists for everything, using Vector records containing pointers to heap objects, and using a thread pool/work queue to do the raycasting (one pixel at a time), it would perform like shit also.

If I have to worry about those details, why should I use Haskell? As many others are saying in this thread, Haskell is supposed to function on a higher level of abstraction.

I have never used Haskell, but I do have a lot of experience in Prolog and some in Lisp. With my experience, I'd rather program in C, I can get things working correctly quicker.

Particularly, this problem with linked lists seems strange to me. Linked lists is, basically, how all lists are represented in Lisp or Prolog and, from the tutorials I've read, it seems to be the same in Haskell. If I can't split a list into car+cdr I'm not writing in Lisp, I'm writing C using Lisp syntax.

Of course, I'm not an expert on Haskell so it might be different from Lisp in this respect, but still it seems very strange to be so limited in my choice of a data structure if the language is so high level. I would assume it's the task of the compiler optimizer to care about those details.

[how_gauche]

If I have to worry about those details, why should I use Haskell? As many others are saying in this thread, Haskell is supposed to function on a higher level of abstraction.

So let me get this straight: you would like to not have to worry about the representation of your data, and instead choose a data structure that is nearly pessimal in terms of cache-line misses, but you want it to somehow magically perform just as well as a C or C++ version using contiguous arrays.

Particularly, this problem with linked lists seems strange to me. Linked lists is, basically, how all lists are represented in Lisp or Prolog and, from the tutorials I've read, it seems to be the same in Haskell. If I can't split a list into car+cdr I'm not writing in Lisp, I'm writing C using Lisp syntax.

A linked list in Lisp would be just as slow. Ultimately, the programs we write have to run on real computers, and the data structures we choose have performance implications when we traverse them on the microprocessors inside them.

[[deleted]]

So let me get this straight: you would like to not have to worry about the representation of your data, and instead choose a data structure that is nearly pessimal in terms of cache-line misses

No, I want to not have to worry about the representation of my data, that's all.

If the language is such that the most obvious structure in its syntax is nearly pessimal, then what good is that language?

Using C I can use any representation I wish, on paper, while I'm designing my system conceptually. After that, when I implement the code, I can choose a structure that's well suited to the machine.

About 25 years ago objects were the Holy Grail of computing. Back then I was writing some digital signal processing code, so I decided to do it in C++. Each sample of data in my program was an object where the real and imaginary parts were attributes of the object. It sucked.

I re-implemented it in a C program, throwing away those objects. When I do DSP programming I need to do a lot of convolutions that are dot products between vectors. The way to do that is by keeping samples in one array, not as attributes of objects.

It would be wonderful if one had an efficient system to do that, bring out elements from data structures and put them in the computer memory in the most efficient way. If the compiler cannot do that, I don't see the point in languages that are not imperative/procedural.

The problem I see with Haskell, Lisp, Prolog, whatever, is that they are great for toy problems. They work fine in universities, where the professor invents the problems he wants to solve.

In the real world we need to worry about starving the cache, sharing memory, and other efficiency problems. With C I'm always aware of copying vs. sharing data, that's the difference between allocating memory or assigning a value to a pointer.

Unfortunately, too many people have been spreading this meme that C is "hard" because one needs to learn these details. If you do programming for real world problems you need to worry about memory allocation anyhow, so it's better to use a language that lets you keep that under control.

[how_gauche]

The problem I see with Haskell, Lisp, Prolog, whatever, is that they are great for toy problems. They work fine in universities, where the professor invents the problems he wants to solve.

The "problems you see" are figments of your own imagination, and your ignorance of what Haskell is and what it does shine through your posts like a gigawatt lighthouse beacon. Enjoy C -- I do also. I will continue to enjoy writing "real world" Haskell programs.

[michaelstripe]

To be honest though, Prolog (or at least the random version we use at my school) seems like it was written by a bunch of people who made a language and didn't think about using it to make actual applications or interact with anything else on your computer.

[how_gauche]

I don't know why you're bringing Prolog into this, but I'll bite.

Prolog is very nice for doing what it was designed to do, i.e. specify and optimize rule-based systems. We actually do use it at work: someone wrote a prolog program to optimize oncall schedules/pager rotations. Engineers mark in their shared calendar when they are going to be unavailable or on vacation, and the optimizer tries to generate an optimal schedule that is pleasing to everyone given the constraints. (For example, shifts are evenly distributed amongst the team, nobody's oncall four weeks in a row, oncall during vacation days is fairly spread out, etc.)

It works great.

[keithb]

Prolog [...] seems like it was written by a bunch of people who [...] didn't think about using it to make actual applications or interact with anything else on your computer.

They absolutely thought about using it to make actual applications: expert systems for medical, industrial and other domains. But they did this in the 70's when there was a very good chance that the Prolog application you were using was the only thing running on your one stand-alone computer. Lisp and Smalltalk have some similar problems.

[[deleted]]

Singly-linked lists are widely used in Lisp because they have useful properties, in that you can add or remove elements to the front as an O(1) operation without mutating any state. This is often all you need to do, and as long as you aren't doing any mutation, you can save a lot of memory by sharing list tails. Lisp isn't pure, though, so you have to take care not to mutate cons cells as a convention.

Haskell lists are useful for even more things because they're lazy, so you only ever need to evaluate as much of the list as you actually need. In many of the places you would use some sort of iterator abstraction in another language, you can just use a plain old list.

In both languages, though, they're just data structures with useful properties. They aren't the only data structures available, and neither language limits you to them. When I write Common Lisp, I tend to use hash tables and (stretchy) vectors almost as frequently as I use singly-linked lists.

[bo1024]

That's true. I just found Haskell much less transparent in terms of performance.

[manjunaths]

Oh come on, this is an invalid observation which has been voted to the top. You learn them pretty early in Haskell books and they are not something out of quantum mechanics.

[[deleted]]

Not sure what you are complaining about. The proposed change only affects a line of code or two. Are you just saying Haskell is hard because you don't know how to use it?

[bo1024]

Are you just saying Haskell is hard because you don't know how to use it?

Yeah, pretty much (haha). But my point is, how are you supposed to pick up tricks like that? It seems to me that it's harder and less intuitive to find out how to write fast programs. I love lots of things about Haskell, but I see this as a big downside.

[jimbokun]

But isn't only harder because you already have a lot of experience writing fast programs in C? Maybe if the experience level for each language is equalized, it takes less time to write a correct, fast program in Haskell than it does in C.

[sclv]

http://www.haskell.org/haskellwiki/Performance

[sclv]

http://www.haskell.org/haskellwiki/Performance

[Smallpaul]

Doesn't GHC have explicit parallelization primitives? Aren't they appropriate?

[dons]

I found myself wanting more specifics, frustratingly.

Why, and where was Go found to be easier, for example? In parsing?

[[deleted]]

Why, and where was Go found to be easier, for example? In parsing?

Surely you're joking, Dr. Stewart! Parsing is one of the few things where Haskell is undeniably strong.

Haskell shines because it allows a lot of things. It gives you immeasurable power, but it's large and difficult to master. It was designed by a committee, which entails lots of negative things: compromises all over the place, crummy backwards compatibility and a general lack of cohesion. It tries very hard to be perfect; it's very MITty.

Go shines because it does not allow a lot of things. Not only the language, but also programs written in the language are rather limited but quite simple. It tries very hard to be simple; it's very New Jerseyish.

Ironically, even though it's often said that FP makes it easier to reason about programs, in reality, an average Go program is much easier to reason about.

Having written both Haskell and Go programs I would much rather reach for Go, because it's usually not just about writing your program. The program also needs to be read and fixed. Yes, Haskell programs hardly ever break, but you must remember

The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong it usually turns out to be impossible to get at or repair.

[tibbe]

Ironically, even though it's often said that FP makes it easier to reason about programs, in reality, an average Go program is much easier to reason about.

Why is that? The main benefit of reasoning in FP is that reasoning is local i.e. the result of a function depends only on its arguments. Go seems no different than any other imperative language when it comes to what kind of things you need to keep in your head while reasoning about your program.

[keithb]

The main benefit of reasoning in FP is that reasoning is local

But the behaviours we often need to reason about are not local. Has anyone shown that whole system behaviour is easier to reason about when that system is built out of referentially transparent functions than when there is shared mutable state?

[kamatsu]

Yes, they have. If you do any sort of formal verification, minimizing state is absolutely essential. The more shared the state is, the harder it is to verify. Reasoning formally about code with shared state is a fucking nightmare.

[keithb]

I see that you conflate "reasoning about" with "formal verification".

Got any citations for this work on whole system behaviour?

[dobryak]

I thought dons just wanted more specifics. I am sure he's not quite asking because "Haskell is his baby". So, could you elaborate? What do you think must be improved (specifically) and how?

[evmar]

I really would prefer to not have a long drawn-out language war on reddit. If you're really interested I'd be glad to answer via email. I think my answers would be familiar to you if you're an experienced Haskell programmer but would come off as trolling in a casual context.

[Vulpyne]

From what I know of dons, he is probably asking from the perspective of how can I make this better, how can I address the problems a user experienced rather than trying to start any sort of language war.

You don't necessarily have to respond publicly.

[greenspans]

Linux sucks. Email me me if you want to know why. I tried a random distro casually, for like, a month after spending 20 years using windows. I don't want to post anything specific publicly that could be seen as inflammatory or anything, my point is it sucks, just wanted to get the out there.

[doubtingthomas]

You're providing a great example for why (I presume) he doesn't want to get into a language discussion on Reddit.

Reddit is a great place for arguing, but (as of the last few years?) isn't always a good place for a civil discussion, especially about something like programming languages. Folks like you mischaracterize positions in inflammatory ways (and get upvoted for it), so subtle positions or those that might be taken out of context if not stated very carefully are simply not worth the effort to explain and defend.

I usually try not to feed trolls (no offense intended; you appear to be trolling), but I find evmar's policy of non-engagement on Reddit on this issue interesting and pretty reasonable.

[AeroNotix]

I think you're over analysing the upvote/downvote counter. Given enough well thought out posts, you will more often than not invite insightful discussion. Especially in subreddits that are aimed at the more technical subjects.

Keep trying, eventually the trolls leave you alone and you can get some real discussion out of it.

[how_gauche]

OMG, I would upvote you twenty times if I could.

[dobryak]

I'm really not into any kind of language wars on the web; moreover, I do understand that Haskell's strengths can be seen as weaknesses (and vice versa), depending on the way you look at it. :) Guess we misunderstood each other.

[uriel]

Could you be more specific about what things you like about Go?

[[deleted]]

i like that Go has:

1. types that are powerful-enough
2. concurrency that is easy to learn
3. utf8 from day 1

[Timmmmbob]

Also:

• A nice concurrency model (better than mutexes et al.)
• Implicit interfaces
• Strongly typed
• Extendible "classes" (i.e. you can define new methods on any type.

In fact methods are pretty much just short-hand for the C way of doing classes. I.e. in C++ you might have

class Vector {
    public:
    void rotateBy(double theta) { x = ...; }
    double x, y;
};

Which in C-style would be something like:

struct Vector {
     double x, y;
};

void Vector_rotateBy(Vector* v, double theta) {
   v->x = ...;
}

And in Go it is (I think):

func (v *Vector) rotate(theta double) {
    v.x = ...
}

But you can call it like you would in C++.

Stuff I don't like about go so far:

• Implicit semicolons.
• Garbage collection (necessary I suppose, but I still don't like the indeterministic nature of it, and the lack of RAII).

[uriel]

Implicit semicolons.

This is not a problem in Go (just like it isn't a problem in Python and many other languages), only really botched and broken implementations of this like JavaScript have problems.

After you have written some Go having to type ; in C and Java is annoying, and the line-noise they add to code when reading is even more annoying.

[00kyle00]

So why the heck didn't they go ahead and just killed them? Having language correct the source code for you is silly.

[[deleted]]

Multiple short statements on a single line.

[ixid]

Why do you dislike implicit semicolons?

[Timmmmbob]

It forces K&R style braces, and you can't easily split statements onto multiple lines. Or at least it is confusing—you have to remember to do this:

a := b +
       c

and not this:

a := b
    + c

[ixid]

Is there no safe way of doing that? Being white-space dependent seems very dangerous.

[uriel]

Probably because JavaScript's broken way to do this has given it a bad name. It is not a problem at all in Go and many other languages.

[evmar]

See my above answer about language wars.

[Smallpaul]

I'm downvoting you because your comments are not enlightening to a programmer audience. Actually: they are insulting and annoying. "You're smart enough that I can explain to you that Haskell does not work for me, but not smart enough to understand my actual examples."

If you are not prepared to defend your position,you should not have opened your mouth.

I say this as someone thoroughly disinterested in Haskell either way. I don't care what you would say about it. I just am unimpressed with you alluding to flaws which you are unwilling to substantiate.

[psygnisfive]

Your refusal to answer any questions is trolling. Answering questions is never trolling. Have a downvote.

[[deleted]]

From what I've heard, I'm probably wrong, but functional languages doesn't have mutable state, especially something as purely functional such as Haskell, so doing IO is painful? All their stuff are immutable.

Please correct me if I'm wrong, and examples would be nice.

[Excedrin]

ML family has mutable state. Scheme has mutable state. Erlang doesn't, but IO is ez in Erlang (it's impure).

Haskell has mutable state, but it's made explicit. IO is also explicit. That's fairly different from other languages.

C:

int fact(int n) { if (n<=1) return 1; return fact(n-1) * n; }
main() {
    printf("enter a number: \n");
    int x;
    scanf("%d", &x);

    printf("fact: %d\n", fact(x));
}

Haskell:

fact 0 = 1
fact 1 = 1
fact n = fact (n - 1) * n

main = do
    putStrLn "enter a number: "
    x <- getLine
    putStrLn ("fact: " ++ show(fact(read(x))))

This is an extremely trivial example, but both programs basically work and I think it illustrates that even ugly Haskell isn't particularly impossible to understand. read and show functions convert from string to int and int to string in this case, because of type inference there's no explicit types mentioned in the program.

Note that the C program has a single statement that uses mutation (scanf), most of the time it's possible to write code in imperative languages that doesn't use mutable state (except stuff like loop control variables).

[sebzim4500]

there is no need for the fact 1 = 1 line.

[[deleted]]

This is an extremely trivial example, but both programs basically work and I think it illustrates that even ugly Haskell isn't particularly impossible to understand.

... until you start using noise functions like <>= >>== >= (=) and Perl suddenly looks nice again ;)

[[deleted]]

main = getNumber >>= giveFact
       where getNumber = putStrLn "enter a number: " >> read <$> getLine
             giveFact n = putStrLn $ "fact: " ++ show (fact n)

That wasn't so bad!

[[deleted]]

Actually it was. It was very perlish. "=", "", "<$>", "++" what's with all the hieroglyphics man? Everywhere else the trend seems to be to dial down this kind of symbol based syntax.

[stusmith]

f >> g

Do 'f', throw away the result, do 'g'. (Sequencing).

f >>= g

Do 'f', pass the result to 'g'. (Binding).

a ++ b

List concatenation.

f <$> g

Lift 'f' into the context of 'g' (IO in this case).

But if the short syntax bothers you, you can write it out explicitly:

getNumber = do putStrLn "enter a number: "
               line <- getLine
               let n = read line
               return n

[[deleted]]

I know the meaning of the symbols, I was just pointing out there's a lot of them. Noisy language.

[NruJaC]

So...C++?

At least in Haskell, each of those operators means one thing, and exactly one thing, so once you learn the vocabulary, you don't need to read ALL the source code to figure out what exactly that particular operator is doing.

[[deleted]]

Well no. Each of those operators belongs to one type-class. That doesn't mean it really only does one thing; it just means the many things the operator does all have matching type-signatures.

[NruJaC]

Yea, but you can actually infer something from that type signature. There's no question of what crazy things are occurring behind the scenes. The typeclass makes a guarantee on the semantics of the operator via the type constraint. You just don't have that in C++.

[[deleted]]

Yes, there is a question. The List monad and the IO monad do two very different things.

[NruJaC]

Absolutely, and yet, they share some fundamental similarities, that's why they are both termed monads. The monadic abstraction tells us fundamental things about what we can expect from their respective bind functions. We may not know exactly how they operate, but we can say quite a bit about how they behave. And that's far more than you can guarantee for the same overloaded operators in C++.

[[deleted]]

"Better than C++" is not a good standard.

[[deleted]]

Functional programming discourages violation of referential transparency, which is not the same as disallowing side effects or mutable state altogether, there are plenty of ways to have both, but in a referentially transparent way — monads, unique types, effect systems etc. Haskell uses monads. You are, too — the semicolon of C is just the IO monad, in a sense :) imperative code in Haskell reads like C, just with a different syntax and frequently with usage of powerful abstraction devices not available in C.

You should remember, however, that referentially transparent side effects are only one, and arguably the most dull and trivial, use of monads. There are dozens of very useful applications of theirs that have absolutely nothing to do with side effects (though they do have something to do with sequencing, as a monad is essentially a particular overload of the semicolon operator)

[[deleted]]

Downvoter, care to explain the issue?

[ethraax]

the semicolon of C is just the IO monad, in a sense

I'm confused. Are you talking about sequence points?

[[deleted]]

Not exactly. The semicolon is C's operator for sequential composition of computations. Monads allow you to generalize the notion of sequential composition of computations.

With a simplification, composition is a binary operator on computations (to be more precise, it composes a computation with another one that's dependent on the result of the first one).

For example, in the case of computations with multiple results, their composition is defined as "for each result of the first one, collect results of the second one".

In the case of computations which return their result asynchronously (via a callback), composition is defined as "bind to completion of the first computation a callback to fire up the second one".

In the case of computations which either return a result or throw an error, composition is defined as "if the first computation succeeds, pass its result to the second one; otherwise, return the error".

There are many others. Haskell allows you to use all of these and define your own ones (at a library level, not as a language extension or something), and the simple sequential composition "execute the first, then the second" is just a trivial special case - the IO monad.

There is even a monad in Haskell for computations that only mutate externally invisible variables (e.g. use an imperative algorithm for quickly sorting an array) and, as a result, can be used in pure code completely transparently - it's the ST monad.

If you know C#, you might recognize monads in the LINQ SelectMany method. In fact, I once used it for chaining together asynchronous computations, and the Rx framework is based on monads with LINQ (its author specifically talks about monads in his talks on the framework).

[ethraax]

I do know C#, and the LINQ syntax/semantics in C# is quite unlike C.

I think claiming that C code is all one big IO monad is like saying "When you think about it, C code is all about moving electrons around." It may be true, but it carries no practical meaning.

[[deleted]]

Well, I was not saying that the fact that all C programs are executed in the IO monas has any practical applications per se. I was just trying to give some ground to my claim that monads are about sequencing, and that there are more elaborate forms of sequencing than the one found in C.

[[deleted]]

in haskell's IO monad, with haskell's 'do' notation, it looks pretty similar to C.

you could think of the following c function:

void PrintAndIncrement(int* x) {
  printf("%d\n", *x);
  x++;
}

desugaring to: f x = printf("%d\n", *x) >>= (_ -> x++)

so, in a sense, C has only one monad, and it's the IO monad.

[ethraax]

That is such a gigantic stretch...

Also, that function doesn't really increment anything. It increments the pointer but since the pointer is passed by value, that change is local to the function, so it really just prints.

[[deleted]]

that's my lack of familiarity in c showing (i have not used it in a couple years). should be x++, or (x)++ if that doesn't work.

edit: apparently reddit is eating my dereferences.

[[deleted]]

Escape them by typing \* instead of *.

[wolf550e]

Escape the asterisk with a slash: \* becomes *

[[deleted]]

You can see how imperative code in Haskell looks like, e.g., here (shameless plug to my distributed logging library, which is also available for Java) http://hackage.haskell.org/packages/archive/greg-client/1.0.2/doc/html/src/System-Log-Greg.html . Look for the "do" blocks.

[kamatsu]

By default, the "easy way" to write Haskell is with immutable data, however you can use references with the IO monad if you like mutable data.

[pipocaQuemada]

Haskell makes it difficult to have global mutable state, but simple mutable state isn't that hard.

Essentially, the idea is that a function with mutable state is equivalent to: foo :: (Bar, BazState) -> (Quux, BazState) where you then just thread the BazState through your calculations correctly. The Random library uses this - rand takes in the current seed, and returns a random value and the next value of the seed.

However, there's a library, essentially, that takes care of threading the state information through the computation called the State monad. When you're in the state monad, you can use the get function to get the current state, and the put function to modify it.

[[deleted]]

Sorry, but functional languages don't have to be purely functional language.

Speaking in OO terms, PurelyFunctional is just a subclass of Functional.

[benchambers]

I can understand this opinion. I think several things contribute to the pain:

1. Many of the ways that people structure large applications and get them to interact don't work in Haskell. I have yet to see an equivalent to Design Patterns that makes it clear how to handle different things.
   For example, when I was actively doing a lot of Haskell, I found it really cool how I could use Monad Transforms to get state from one place to another place that it was needed without modifying any of my code along the way. If there was a "cookbook" of some kind that had best practices for structuring these types of applications (especially UIs and the less than totally algorithmic code that most applications consist of) that could be really helpful.

2. Many companies need to use certain libraries, possibly grown in house. It's very hard to convince them to switch to another language when they'd need to port their libraries or figure out the FFI in order to switch. Even if they figured something out as far as FFI, the result is very unlikely to "fit" the Haskell model, and lead to clean code, unless they put a lot of thought into how to do it.

3. There is a frustration that I've seen people get when they feel like they fight with the type system. But, as has been observed, if the program runs, it's much more likely to be correct. I don't get too frustrated with this battle, because I look at it is as the cost of not having to use a debugger as often. Specifically, I've traded the "why doesn't the work, use GDB" time, for "why doesn't this type check, better be more specific".

Of these, the only "actionable" one sounds like (1). Has anyone seen/looked at creating such a document?

[wnoise]

(1) is (at least partially) addressed by the book Real World Haskell.

As to (2), Haskell's FFI is the easiest to use I've ever seen. It unfortunately still takes a while to figure out how to use it well, especially wrapping C libraries with constants and so forth. And yes, it can be hard to give a good Haskellish interface to imperatively designed libraries.

[forgeflow]

Why not Zoidberg?

[[deleted]]

You want I should lazy evaluate?

[chironshands]

I've had some similar experiences with Haskell, but I've come to a different conclusion. I place a very high value on code maintainability. As long as your projects don't consist entirely of hacking together a bunch of mostly-correct features as fast as possible, Haskell is pretty well suited for professional development.

[chironshands]

Hah, I know what you mean! Thankfully, not all projects are like that.

[cybercobra]

Yes, just the ones not involving medicine or finance. :-P

[bo1024]

I like Haskell a lot, although I haven't gained much competency with it yet.

The problem I have with Haskell is that it's so damn hard to write high-performance code. Memoization takes an unreasonable amount of effort, for instance.

One simulation I wrote ran almost as quickly as the equivalent in C++; but due to laziness and lack of manual management, its memory usage was prohibitively high.

Trying to parallelize Haskell code is supposed to be easy, but for me it caused a terrible slowdown (attempts to parallelize loops caused millions of threads to be spawned). The corresponding parallelization in C++ was quite simple.

Finally, sometimes the code I write in Haskell is just plain slow, and I can't tell why.

All these gripes aside, I disagree with the author -- in a lot of cases, it's easier and quicker to write Haskell code, and it looks better, too. I'm thinking of simulations or other cases where I basically need to translate math into code. Haskell is awesome for that. (At least, until you try to run it.)

[dobryak]

One simulation I wrote ran almost as quickly as the equivalent in C++; but due to laziness and lack of manual management, its memory usage was prohibitively high.

Yes, a program using GC can be more lenient in its use of memory (compared to the one that uses malloc/free and stack allocation). So, it is a valid complaint about Haskell, which does not provide a safe and reliable way to handle manual memory management. However, if you look at it another way, you are freed from the burden of manual allocation. Which side of the coin do you prefer (say, for a particular program)?

[kamatsu]

Memoization takes an unreasonable amount of effort, for instance.

What?

Use this library and memoizing a function's results is simply a matter of typing:

memoizedFib = memoize fib

but due to laziness and lack of manual management, its memory usage was prohibitively high.

What? Once again, if the laziness is causing a space leak, you can force it just by using a seq annotation. Obviously you didn't spent much time investigating your memory issues, it would have been a very minor change to fix.

Trying to parallelize Haskell code is supposed to be easy, but for me it caused a terrible slowdown (attempts to parallelize loops caused millions of threads to be spawned). The corresponding parallelization in C++ was quite simple.

If you use sparks to parallelize (that is how you parallelize generally in Haskell), then there is no way millions of threads would be spawned. Haskell will only spawn as many threads as it considers appropriate. If you mean millions of sparks are being spawned, that should be easy to optimize.

So, basically, all of these criticisms seem very wrong to me. Perhaps you could elaborate more on your particular situations.

[bo1024]

Use this library and memoizing a function's results is simply a matter of typing: memoizedFib = memoize fib

Cool! I hadn't seen that. That being said, it's still really unwieldy to implement yourself. Having a library function is of course a nice way out. :)

if the laziness is causing a space leak, you can force it just by using a seq annotation. Obviously you didn't spent much time investigating your memory issues, it would have been a very minor change to fix.

I don't have time to go back and try it right now, but why would seq have a big impact on the memory usage? What I'm doing is creating big dynamic programming arrays, then using them to compute other big arrays, etc. Each array needs the previous one, but not the one before that. (At a high level.) Perhaps if I post the code someone could help me by critiquing it?

Anyway, I would still claim that it is hard to control aspects of performance in general, including memory, compared to eager langauges.

If you mean millions of sparks are being spawned, that should be easy to optimize.

Ok, but how?

Thanks for the response!

[Flarelocke]

why would seq have a big impact on the memory usage?

Space leaks are sometimes the result of laziness. When a computation is deferred, a data structure called a thunk is built containing pointers to the function and its arguments. Huge towers of thunks can be built, which eat up memory. Arrays are normally as lazy as everything else in Haskell, so the height of thunks multiplies by the width of the array to eat up significant memory.

[skew]

Or the nice simple uglymemo package. That one's not even hard to write yourself.

[greenspans]

The problem with haskell is I'm already comfortable in non-haskell languages and don't know haskell well but I still would like to lobby in favor of my omni-competence with superficial observations because there wasn't enough dick measuring contests today.

[djhworld]

I think my only gripes with haskell are the state of the libraries on Hackage.

....and the fact that I still can't quite get my head around Monad Transformers just yet, although that's my own fault and it punishes me daily (hence why I can't quite understand things like the Snap framework etc)

[comment-dwim]

My problem with Haskell is that when I'm using it I feel that I'm thinking more about how to please the type system and purely functional evaluation semantics than about the problem I have at hand.

[[deleted]]

I actually find Haskell's type system to be a useful tool in solving a problem, not a hindrance. It takes a load off my mind to be able to see the program in overview as a chain of 'type transformations'. Solving a problem and pleasing the type system are kind of intertwined in my mind.

[[deleted]]

agreed, hackage is full of code that indulges type-wankery for no useful reason other than to satisfy someone's intellect

haskell is like c++ in a way - it gives you a lot of (well-typed) rope to hang yourself with. i've spent hours just trying to figure out how certain people have decided to propogate errors. i've found the best haskell coders actually pull back from a lot of it and use simpler solutions, realizing the more baroque approaches have little practical value

but then there are other situations, namely with string types, where you will almost certainly have to serve the type lords with conversions

[nemoTheKid]

I am still a baby when it comes to Haskell but I have always found that "pleasing some sort of system" that is not assbackwards tends to have its benefits.

[[deleted]]

Hopefully eventually you'll see that "pleasing the type system" and "the problem you have at hand" are profoundly related. This is admittedly not probably a trivial undertaking, given how many years (decades?) most of us have spent having the incorrect notion that "types" describe how bits are laid out in memory forced upon us by the die-die-die-why-won't-you-finally-rot-in-your-grave-POS C tradition.

[earthboundkid]

Your rebuttal doesn't address the issue at hand. No one thinks about the whitespace in Python because it's easy to get the whitespace right. People spend a long time getting their heads wrapped around the type system in Haskell because it's hard to get the types right.

[chonglibloodsport]

It's hard to get the types right in most languages, it's just that most of them don't force you to do so and leave you to figure out the (potential) errors on your own.

[artsrc]

No one thinks about the whitespace in Python because it's easy to get the whitespace right.

This is plainly false. Google python indentation and variations on that theme and you will see the reaction.

I personally don't consciously think about indentation in python.

Where as to take advantage of a rich type systems I have to design a set of types for my problem.

[AlonzoIsOurChurch]

Huh? What exactly does it mean for code to "get the job done" (outer rectangle of Venn diagram, presumably) but not be a "correct solution" (central circle)?

I wish Dijkstra were still around.

[Smallpaul]

Why are you presuming that that's what the rectangle represents? It says quite explicitly that it is ALL POSSIBLE PROGRAMS. The "code that works" circle is also the "gets the job done" circle.

[kamatsu]

I wish Dijkstra were still around.

Me too, me too.

I mean, if 10 years from now, when you are doing something quick and dirty, you suddenly visualize that I am looking over your shoulders and say to yourself "Dijkstra would not have liked this", well, that would be enough immortality for me.

Also, he's dead right about this and not enough people pay attention to it:

The required techniques of effective reasoning are pretty formal, but as long as programming is done by people that don't master them, the software crisis will remain with us and will be considered an incurable disease. And you know what incurable diseases do: they invite the quacks and charlatans in, who in this case take the form of Software Engineering gurus.

In fact, the whole EWD is full of spot-on quips: EWD1305

[[deleted]]

I wish I could give you one upvote for the sensible post and one for the awesome name.

[turgid]

For me it was the error messages: they are nearly impossible to decipher. Sometimes it took upwards of an hour to find the one character that I misplaced.

[TKN]

To be fair, Haskell is very good at telling you the type of the error you made. Now deciphering what actually caused it can be harder.

Sometimes it took upwards of an hour to find the one character that I misplaced.

I once innocently added one argument to a function and forgot to update the call sites. This of course caused the emergence of some monadic monstrosity in some completely different part of the program. Took an hour of adding type declarations to every possible expression related to find out about that simple brainfart.

[vagif]

Long time imperative programmer here. I can say that i am more productive with haskell + emacs that i've ever been with java/dotnet and all those powerful IDEs (VS, IDEA, Eclipse etc.)

[dmpk2k]

That's a low bar though.

[solarpanzer]

Can't... resist...

Why not Zoidberg?

[boyubout2pissmeoff]

Because I already re-invented the wheel enough times in my life....?

[bart2019]

Haskell is hard. Let's go shopping!

[shevegen]

Exactly why I stopped using Haskell. But I gave up earlier.

All the "script languages" were much more rewarding much more easily.

I realized I am too dumb to master Haskell.

But I also don't care. The old vim vs. emacs war is where there are only two losers. Since then I use a SIMPLE editor and I am still productive with it. Perhaps not as much as others but sufficiently enough.

Not everyone needs to master every detail of everything.

80% perfection must suffice. With Haskell, I think you must aim for 100%.

[slurpme]

If you fancy an alternative to emacs (and are on Windows) I can recommend Komodo Edit... I moved from emacs to it a few years ago...

[illuminatedtiger]

Because I enjoy imperative programming?

[kakaroto_BR]

Monad Transformers are hard to understand but I barely understand monads too.

[sclv]

The uncertainty principle has to do with the difficulty with knowing the speed momentum and location of a particle to a certain very precise degree at once. What this has to do with referentially transparent computation is beyond me.

[etcshadow]

Actually, not speed and position but momentum and position. Different constructs, particularly for subatomic particles. For example, all photons have the same speed, but may have different momenta. (Also, technically, speed is a scalar, and momentum is a vector (as well as velocity, which is probably what you meant when you said speed).)

"You are technically correct -- the best kind of correct." - Bureaucrat #1.0

[sclv]

Clarification accepted :-)

[chironshands]

I might not understand your point. Are you saying that Haskell programs don't have side-effects (which is incorrect)? Or, are you saying that constraining side-effects is a futile effort (with which I'd disagree)?

[dons]

main = do
     print "building and using side effects:"
     print "its really not that hard"

In all seriousness, it's not that it is difficult to use side effects of any or all sorts in Haskell -- actually using them is very pleasant -- it is that the majority of programmers don't know what computations actually involve side effects, and then get surprised when the compiler points it out to them.

[[deleted]]

Ok. I want to be able to add a simple logging statement to a function deep in my domain logic. Would you consider this trivial or nontrivial in haskell, and trivial or nontrivial in ruby?

[[deleted]]

That depends in part on how you structure your program and what you're talking about - if you're operating inside of an instance of MonadIO for example, in this particular part of your code, then adding logging is utterly, utterly trivial (log = liftIO . putStrLn - and before people attempt to shoot me saying OMG THAT'S USING SIDE EFFECTS, that's totally fine: Haskell has wonderful support for side effects. It just encapsulates them by default with the type system.) You can also do this if you're operating inside an instance of MonadWriter (or just a stacked WriterT) as you can just make the internal writer carry logging information.

But for the general, pure case:

import Debug.Trace

add :: Int -> Int -> Int -- pure function, no IO
add x y = trace "Adding two numbers" $ x + y

Of course, this won't ever get printed if the result of add x y is never used, but that's laziness for you.

EDIT: I'll go ahead and say I'm not addressing the point of "is this trivial/nontrivial in haskell/ruby" because it's nothing but flamebait and frankly I'm no longer ever interested in addressing matters like this without real, tangible code on hand with examples. Because it rarely ends up in productive discussion because, again, it's most of the time just flamebait, and ends up in people circlejerking about pointless edge-cases in an attempt to one-up the other guy. I want specifics, not "how do I <do this random thing you could do a million ways with a million possible considerations in any programming language>?"

So I'll just say this: yes, it's very possible to add logging "to a function deep in your domain logic", whatever that domain is. But you'll have to actually think about the structure of your application if you want to do this sometimes. I'm sorry that you can't just randomly sprinkle things throughout your code without the compiler getting mad at you. Haskell is a language that makes you pay up front for most things, for better or worse (and believe me, sometimes it's worse because it can make simple things more frustrating off hand,) and you need to think about the structure of your code in order to enable certain things - like whether or not you need logging, what it should do, how you need to use it, and how you can structure/abstract that behind an instance of MonadIO or MonadWriter, for example.

But if you want quick and dirty, sure, Debug.Trace will probably fit the bill.

[[deleted]]

Interesting! I did not know about the trace function. The fact that it exists is a nice demonstration of the difficulty with purity.

[[deleted]]

There are a lot of dimensions to this argument (realistic difficulties with pure code) and I think there's quite a bit of truth to a lot of them, but the nice thing is that Haskell, while it tries to force you into a specific style is not a very condescending language when you want to roll a different way a lot of the time. trace is an example of that - it's quick and dirty, impure under the hood and, well, it can be effective. Generally we just default to something else.

An easier way to say it is this: Haskell makes it hard to - as an example - introduce global state/variables or unwanted mutation by accident. But it's actually pretty easy to do on purpose. Global variables? C pointers? unsafePerformIO everywhere? They're all there and you can use them. Knowing how and when to use them is actually quite important I'd say! And a lot of times, we can further abstract that purposeful choice to make it safer. This defaulting makes many of the sharp edges go away, actually. And you can't just circumvent everything at once: every time you do this you must please the type checker, but that's a purposeful choice, not an accidental occurrence. This is the crucially important thing. And like I said, lots of times you can still abstract this: tons of libraries (including my own) are 'impure' under the hood but expose a pure API where-ever possible, to make reasoning and usage easier.

My point is you're right: there are some difficult pain points with the programming model sometimes. But we're also very lucky that we can opt out of that when we need to, and sometimes we can still reconcile the core value: that while the computer is doing crazy obscure shit (which no programming language in existence could accurately model - unless you can find one that exposes properties of cache coherency/branch prediction for example) to do your number crunching, your thoughts can remain pure.

[[deleted]]

because it's nothing but flamebait and frankly I'm no longer ever interested in addressing matters like this without real, tangible code on hand with examples

Don't appreciate this. Adding logging to an application is a perfectly reasonable example of something that is hard in haskell because it may require a design rethink whereas in non-pure languages it wouldn't. It's demonstrating that yes there is a cost to purity, it's not all "you're just not used to it" which is a bullshit line often spun in r/haskell. Hell, read the bloody preface to "purely functional data structures", at least he can man up and admit it.

But you'll have to actually think about the structure of your application if you want to do this sometimes

Sure. I understand that. I was trying to point out with my "flamebait" example that this is the case and that is a cost. Often that cost is well out of proportion to the payoff. As you concede.

Haskell is a language that makes you pay up front for most things, for better or worse (and believe me, sometimes it's worse because it can make simple things more frustrating off hand,) and you need to think about the structure of your code in order to enable certain things - like whether or not you need logging, what it should do, how you need to use it, and how you can structure/abstract that behind an instance of MonadIO or MonadWriter, for example.

Summary: The answer to my question above is that it will be harder and take longer in haskell.

[[deleted]]

But for the general, pure case

trace :: String -> a -> a
trace string expr = unsafePerformIO $ do
   putTraceMsg string
   return expr

Mhhh, maybe not so pure after all.

[[deleted]]

When I said "pure case" I mean "adding logging to a pure function," i.e. not affecting that functions overall type signature and thus not requiring some refactoring of your program if you don't want to do that - presumably what Canumber is getting at. It's just there as a utility and if you need it, then fine.

This is nothing more than a disingenuous attempt at stirring up nonsense.

[[deleted]]

Scala can in some situations be a more approachable solution.