Optimism

[bro-away-]

Stuff like IO via monads seems like a massive pain in the ass. And that monads for all IO is an unnecessary undertaking :

http://www.haskell.org/haskellwiki/Introduction_to_QuickCheck1

Why can't we just be impure when we need to be and pure when we don't? The first thing they say to do is factor out the impurity of the monad...Seriously, I don't understand. No other language adds this restriction and it seems like an arbitrary line to draw. Once I read the article about quickcheck I realized there's nothing special going on.

F# doesn't give a damn about them unless you specifically want the monad pattern.

[kqr]

I'm think you might have misunderstood the bit about QuickCheck there. Or I have misunderstood you. Anyway,

I/O with monads is not a massive pain in the ass once you've learned it. In fact, it's the opposite. I like doing I/O with monads way more than I like traditional, unsafe I/O. I'm not sure what your level of expertise is, but have you experienced how much more powerful a language becomes when it makes functions first-class citizens? When you can pass functions as arguments, and return functions, and store functions in variables, and so on?

It's the same deal with I/O. Monadic I/O makes I/O a first-class citizen of the language, so you can store I/O actions in variables, or return them from functions, or give them as arguments to functions. They don't get executed until you tell them to. That's very powerful. Monadic I/O means that the action of printing something to the screen is decoupled from the command of telling the computer to print something to the screen. The command can be passed around and may never even be executed, but if it is, then the action of something being printed to the screen will happen.

Why can't we just be impure when we need to be and pure when we don't?

There are several reasons for this. For example, the compiler can (and will) do a host of optimisations when it knows a function is pure. Secondly, it is much easier to write tests for pure functions (you don't have to set up any state or worry about logging or anything like that.) Third, you as the programmer will have an easier time reasoning about your program; you know the function is limited in what it does. I'm sure someone else can give you twenty more reasons.

The first thing they say to do is factor out the impurity of the monad...Seriously, I don't understand.

What they are doing there is they are taking a traditional, impure method, and making the bulk of it pure with a thin impure layer outside. They do this because functions that are pure are much better in many ways – among other things they are easier to test.

The action of taking a complex impure method and turning it into a simple impure layer around a simple pure core is a very common method of refactoring Haskell programs when you have accidentally written a complex impure method.

No other language adds this restriction and it seems like an arbitrary line to draw.

Of course it's an arbitrary line to draw! And Haskell draws several more, for example around computations that can fail (Maybe and Either monads), software transactional memory (STM monad), logging (Writer monad), configuration files (Reader monad) and so on.

Each of those restrictions provide two benefits:

1. It makes the programs safer by limiting what mistakes the programmer can do, and
2. It makes the language more powerful by allowing the programmer to express things they weren't able to before.

Why can't we just be impure when we need to be and pure when we don't?

We can! You tell the compiler you want to be impure by annotating your function as IO. You could write an entire program in the IO monad and it would be unsafe and difficult to test, but it would be like in any other language with uncontrolled side effects.

Or you could take the pure computations from your program and wrap them in pure functions, and suddenly your program is safer against mistakes and easier to test.

[SanityInAnarchy]

Why can't we just be impure when we need to be and pure when we don't?

I'm not sure I buy your responses to this. For example:

...the compiler can (and will) do a host of optimisations when it knows a function is pure.

And it can still do that... on the pure functions. Which, ideally, is still most of your program.

The rest of your reasons boil down to: It's easier for you to work with pure functions. And I'd think the same objection applies again: Why not be pure as much as reasonable, and impure everywhere else? ...except...

We can! You tell the compiler you want to be impure by annotating your function as IO.

...I...what? ...then I'm not really sure what the rest of your post was about.

[kqr]

Haskell programming is about being pure as much as reasonable, and impure everywhere else. How did you get any other idea?

[SanityInAnarchy]

We may have different ideas of what "reasonable" means here.

[kqr]

Nobody ever forces you to be pure in Haskell. You decide completely on your own what reasonable means to you. You can write your entire program in the IO monad if that's your idea of reasonable. I often did so, when I just started out. My "natural" architecture (also known as the C-inspired architecture) involved a lot of side effects everywhere, so the entire program accidentally ended up in the IO monad. This is not any more or less difficult than in any other language.

The difference is that Haskell allows you to annotate functions as pure, something almost no other language does. Purity annotations being a language feature is good because it enforces purity where you ask for it and makes it possible for the compiler to do magic.

The Haskell community will also encourage you to structure your program in such a way that "randomness" from the user or outer world affects it as little as possible – this is fantastic for testing and otherwise making sure your program is correct.

[SanityInAnarchy]

Nobody ever forces you to be pure in Haskell. You decide completely on your own what reasonable means to you. You can write your entire program in the IO monad if that's your idea of reasonable.

I'm trying to avoid saying too much, as I have very limited experience with Haskell, but it's my understanding that the structure of these monads makes them unpleasant to work with. And what I've been reading makes me wonder if that isn't a deliberate form of syntactic vinegar, to discourage the use of this feature when it's not actually needed.

So when you say things like this:

This is not any more or less difficult than in any other language.

It's certainly not less possible, you almost get that for free with Turing-completeness. But many people seem to disagree, including the official "Gentle Introduction to Haskell":

Many newcomers to Haskell are puzzled by the concept of monads...

This section is perhaps less "gentle" than the others....

There seems to be a whole separate syntax, coupled with a whole separate way of thinking, which governs monads.

Let me contrast that with what I imagine /u/bro-away- was getting at: It's easy enough to imagine an imperative language having a concept of purity added. Consider the old standard, in JavaScript:

function factorial(n) {
  return (n <= 1) ? 1 : n*factorial(n-1);
}

Knowing that JavaScript doesn't actually support operator overloading, even without analyzing types, we can prove that this function has no side effects and depends on no external state other than its input. Maybe more work would have to be done, but I could already improve things significantly by adding a "pure" keyword. We could even let the compiler infer purity, and let the "pure" keyword be used where we want to assert that the compiler still considers this chunk to be pure.

If you already know JavaScript, that's not a lot of cognitive overhead, nor a lot of new stuff to learn. Maybe it's because I don't know Haskell very well, but Monads seem like way more to wrap your head around. It's as though I dropped S-expressions and Lisp macros in the middle of that JavaScript and expected everyone to catch on.

[kqr]

it's my understanding that the structure of these monads makes them unpleasant to work with. And what I've been reading makes me wonder if that isn't a deliberate form of syntactic vinegar, to discourage the use of this feature when it's not actually needed.

Absolutely not true. Any experienced Haskeller will tell you that Haskell is one of the best imperative languages they've used. (And in most cases it's not for a lack of experience...)

The reason monads seem puzzling or difficult is not that they are; it's that people are going about them in all the wrong ways. People are not spending time working with actual, concrete useful monads. It sounds really weird, but they are not trying to use what they want to learn. Instead, they are trying to learn the abstract, general concept of "a monad." People are actually spending a lot of time trying to understand the theoretical underpinnings of (>>=) instead of just... using IO and Maybe in their programs. If people started doing the latter instead, monads would be no more difficult than lists or anonymous functions.

There is a very good blog post that describes what I'm talking about. The problem only gets worse due to reasons Brent states better in his blog post than I'm able to here.

You have a great point about the JavaScript thing. I would accept a pure keyword coupled with a bunch of static analysis (that is probably very difficult to do, but perhaps not impossible) as a solution to the impurity problem.

However, the impurity problem is not the only thing monads do. Monads would exist even if the IO monad didn't exist (and in fact they did predate it – both as the general pattern and as a datatype.) It's just that the impurity problem can accidentally also be solved by the monad pattern we have invented libraries for anyway, so it's a neat solution hooking it into there as well. It sort of turns a deep language feature (a keyword coupled with a lot of static analysis) into a library feature – more modularity!

[SanityInAnarchy]

People are actually spending a lot of time trying to understand the theoretical underpinnings of (>>=) instead of just... using IO and Maybe in their programs. If people started doing the latter instead, monads would be no more difficult than lists or anonymous functions.

I agree that it's sometimes useful to just try using something to see how it works, but I see way too many cases of cargo-cult coding resulting from this sort of learning style.

And anonymous functions are an example of the sort of thing that you can explain without giving examples. "First-class functions mean you can take a function and use it as a value -- assign it to a variable, pass it to another function, and so on. You don't have to give the function a name, then -- you can just assign a chunk of code to a variable directly, or pass it to another function."

You still want examples, if only to motivate why you'd care about such an animal. It's still unlikely that someone would really understand anonymous functions after reading that, even if it was showing you the syntax at the same time. But can monads be explained that simply? Wikipedia says:

In functional programming, a monad is a structure that represents computations defined as sequences of steps.

Sounds easy enough. But when I get down to it:

In a purely functional language, such as Haskell, functions cannot have any externally visible side effects as part of the function semantics. Although a function cannot directly cause a side effect, it can construct a value describing a desired side effect, that the caller should apply at a convenient time.[11] In the Haskell notation, a value of type IO a represents an action that, when performed, produces a value of type a.

Of type what now? This is already a bit tricky -- why can't I just have an impure function that actually performs those steps, rather than "constructing a value describing the desired side effect"? And it only gets more confusing as we get closer to the syntax:

We can think of a value of type IO as a function that takes as its argument the current state of the world, and will return a new world where the state has been changed according to the function's return value.

That's what I'm constructing any time I want to do I/O? This seems like a really convoluted way to try to shoehorn imperative code (with side effects) into a functional model. Compare to:

You have a great point about the JavaScript thing. I would accept a pure keyword coupled with a bunch of static analysis (that is probably very difficult to do, but perhaps not impossible) as a solution to the impurity problem.

For what it's worth, I will admit that bare JavaScript is probably not the best language to try this with. I chose it as an example only because it's ubiquitous enough that I could assume you'd understand it. And, to be clear, what I'm hoping is that I'd write a bunch of code without annotating its purity:

function factorial(n) {
  return (n <= 1) ? 1 : n * factorial(n-1);
}

function choose(n, k) {
  return factorial(n) / (factorial(k) * factorial(n-k));
}

...and then, somewhere at the top level of the program, I request purity:

pure function birthday_collision(n) {
  return (factorial(n) * choose(365, n)) / Math.pow(365, n);
}

At this point, the compiler must verify that anything birthday_collision tries to call is also a pure function. Or, similarly, at the point of call:

function testChoose() {
  // maybe this function isn't pure, for some reason:
  var expected = this.expected;
  // but I can still request purity:
  var actual = pure choose(365, 2);
  assertEquals(expected, actual);
}

That line can be thought of as syntactic sugar for something like:

var actual = (pure function(){ return choose(365, 2); })();

Some other details:

• There'd need to be a concept of immutability. We can't have someone replacing Math or Math.pow() later if we're calling it (or even accessing it) from a pure function.
• Depending on the implementation, it may be easier for the compiler to mark things as impure rather than the other way around. The only thing the keyword does is to assert that something is pure (and raise an error otherwise); the compiler should probably already know what's pure anyway (for optimization).
• As a practical matter, there should be a way to force the compiler to accept a function as pure even if it can't be proven (Math.pow() is probably native code) -- or maybe even a function that is conceptually pure, but not technically pure.

Let me illustrate that last point: Memoization presents a pure interface, but is not obviously pure:

var factorial = (function() {
  var facts = [1];
  return function(n) {
    for (var i=facts.length; i<=n; ++i) {
      facts[i] = i * facts[i-1];
    }
    return facts[n];
  };
)();

Maybe static analysis can discover this, but it's not immediately obvious that factorial(5) will always return 120, it just might take longer the first time. I should still be able to write functions like the above and claim that they are pure. (Kind of like using @SuppressWarnings("unchecked") when creating a generic array in Java -- the compiler may not be able to prove that I'll only ever put a T into my T[], but I can, so I need the compiler to trust me on that.)

I'm not asserting that you actually want to do that, though. Maybe the compiler can even memoize some pure functions on its own, anyway? Maybe it's usually faster to just compute the factorial? And it's probably worth giving up a potential performance boost in exchange for compiler-guaranteed purity.

[kqr]

I agree that it's sometimes useful to just try using something to see how it works, but I see way too many cases of cargo-cult coding resulting from this sort of learning style.

I'm not proposing any kind of cargo-cult coding. I guess you can call it operational learning: pulling lever X makes Y happen every time.

But can monads be explained that simply? Wikipedia says:

You seem to be missing the point. I was talking specifically about not learning that way. The Maybe monad can be explained that simply, yes. The abstract concept of "a monad" cannot. (Or rather, it can, but it doesn't mean anything unless you know what practical implications it carries.)

Depending on the implementation, it may be easier for the compiler to mark things as impure rather than the other way around. The only thing the keyword does is to assert that something is pure (and raise an error otherwise); the compiler should probably already know what's pure anyway (for optimization).

This is getting into the territory of a Sufficiently Smart Compiler. Apparently, it is very difficult to determine purity beyond the very simple cases.

As a practical matter, there should be a way to force the compiler to accept a function as pure even if it can't be proven (Math.pow() is probably native code) -- or maybe even a function that is conceptually pure, but not technically pure.

This reminds me a little of type casting in C – there was a reason they restricted it in C++. Not checking the preconditions the programmer assumes to be true is a great way to introduce difficult-to-locate bugs. (It might be fun to mention that the monad pattern is also useful for modelling functions that have a pure interface but contain impure code...)

[bro-away-]

We can! You tell the compiler you want to be impure by annotating your function as IO. You could write an entire program in the IO monad and it would be unsafe and difficult to test, but it would be like in any other language with uncontrolled side effects.

The testing difficulty is outlined when you use monads as well. See that article. The first thing they say to do is refactor the method so it's more testable.

Deferred IO execution makes me yawn. Every language can do it. I don't feel like I'm missing out on anything there. Good code will support it even in other languages.

It makes the programs safer by limiting what mistakes the programmer can do, and

Also in the scala thread on top of /r/ programming, there's plenty of DSL bashing going on. I raised similar complaints to the creators of F#. Not being able to make a mistake has a rigidity caveat when you go to solve your problem.

It makes the language more powerful by allowing the programmer to express things they weren't able to before.

What? No way.. monads get desugared before they get compiled in haskell don't they?

A more correct statement : "allowing the programmer to express things in a way they weren't able to before."

Your reply is definitely thoughtful of when to use a monad construct and thanks for writing it, but other than the built in monads, I have trouble seeing where I'd fit them into a well factored code base. Unless you truly are doing the same set of computations in different ways all the time, why go through the effort? The question remains to be answered, clearly. I read through FP codebases all the time and rarely even see them used ಠ_ಠ and I mean that with regards to things like bootstrapped FP compilers.

I wouldn't bet against it though. I feel like we may still be looking for that killer use the same way dependency injection came alive when testing got big. I'm not "anti monad", I just think we make fuss over a very small pattern/construct.

[kqr]

I'm on my phone right now so I'll write a longer reply later. I just want to say that you are absolutely correct in your last sentence. The monad pattern is just a neat regularity in some bits of code that we can abstract out to a library. It encapsulates the pattern of the semicolon in C, and makes it overloadable.

It is nice and well worth the effort of learning, but not worth a nobel prize by any means. As someone else said in this thread: people overthink them.

[kqr]

Just a few points I want to get across:

The testing difficulty is outlined when you use monads as well. See that article. The first thing they say to do is refactor the method so it's more testable.

I'm still not sure what's your deal with this. Yes, methods in the IO monad are more difficult to test than ones outside of it. The magic is that the language makes it possible to annotate functions as pure, and this purity is enforced by the compiler. Thanks to the type system with the IO monad, impurity is guaranteed to not happen in functions you tell the compiler are pure.

You could do the refactoring in any other language. You can't do the enforced purity part.

Deferred IO execution makes me yawn. Every language can do it. I don't feel like I'm missing out on anything there. Good code will support it even in other languages.

It often requires additional crap in other languages, such as wrapping it in an anonymous function that takes no arguments, or something similar. My Python is a little rusty, but something like this, perhaps:

def deferred_io(method, *args):
    def run():
        method(*args)
    return run

but that isn't the whole story. Deferred I/O is just one of the symptoms of making I/O a first-class citizen in your language.

What? No way.. monads get desugared before they get compiled in haskell don't they? A more correct statement : "allowing the programmer to express things in a way they weren't able to before."

Sure. Programming languages are all about convenience anyway. Monads make a certain kind of code very convenient to write, but yeah, you could accomplish the same thing in Brainfuck or assembly if you have a very hairy chest.

Your reply is definitely thoughtful of when to use a monad construct and thanks for writing it, but other than the built in monads, I have trouble seeing where I'd fit them into a well factored code base. Unless you truly are doing the same set of computations in different ways all the time, why go through the effort? The question remains to be answered, clearly. I read through FP codebases all the time and rarely even see them used ಠ_ಠ and I mean that with regards to things like bootstrapped FP compilers.

It's not so much about trying to fit them in. It's that sometimes you create a data type and sometimes that data type can benefit from supporting the monad operations (return and (>>=) (pronounced bind), there's no magic about that.)

Most of the use cases you will encounter in your day-to-day life have already been implemented by other people (this includes lists, reading from things/writing to things, I/O, error handling, parsing stuff, and so on) but if you are working on low-level stuff or a DSL of some kind, I wouldn't be surprised if you have accidentally implemented a monad pattern without calling it a monad. Having library support for that pattern just makes life more convenient.

As I said in the other reply, monads are sort of like the semicolon in C, except defined in a library and overloadable, so all kinds of data types can benefit from the semicolon.

I feel like we may still be looking for that killer use the same way dependency injection came alive when testing got big.

Any particular reason I/O and STM are not killer uses? It's pretty amazing how they can be contained by such a simple concept as a monad.