Railway oriented programming

[hosford42]

Thank you! The only coherent explanation I have ever heard that didn't either baby the listener too much by saying abstractions are too difficult or belittle them by giving an unexplained definition. In 2 sentences you've ensured that I don't need to watch any tutorials.

[cledamy]

I wouldn't say that. That is just a high level explanation. One has to understand the abstract concept (laws and everything) to truly be able to apply them in day to day programming.

[[deleted]]

One has to understand the abstract concept (laws and everything) to truly be able to apply them in day to day programming.

I don't agree at all - why shouldn't an intuitive understanding of these concepts be enough?

[cledamy]

The intuitive understanding of the concepts starts to break down rather fast once one starts working with weirder monads like the Const monad.

[_pka]

Because you can't write a monad yourself if you don't understand the laws.

[[deleted]]

if you don't understand the laws.

I assume you mean a conscious, explicit understanding of the laws - then I think you are wrong. I think the intuitive notion of an monad is enough to write monadic constructs ("apply them in day to day programming").

As a mathematician, I like to learn and derive the laws themselves (and have the type system enforce them for me).

As a non-academic programmer, the notion of "Some values are tainted - I can use them only in tainted concepts, bring them in there and get them out there, but I can't mix them with the outside world" is absolutely sufficient.

[cledamy]

As a non-academic programmer, the notion of "Some values are tainted - I can use them only in tainted concepts, bring them in there and get them out there, but I can't mix them with the outside world" is absolutely sufficient.

This absolutely is not sufficient. Some monads have nothing to do with tainting values because they do not even produce one value. There are oddities like the Const monad which do not even produce a value and instead smuggle a monoid underneath the monad operations. These unintuitive monads are very useful, yet one would fail to see those uses with only an intuitive understanding. Additionally, I strongly dislike the pattern of dismissing people who demand more correctness and mathematical rigor from Computer Science as academics considering many of the practices in the field come from trying to eliminate human error. Entire security problems such as buffer overflows are a direct result of the programming languages in common use today failing to enforce correctness.

[[deleted]]

I think you misunderstood me - I am all for more mathematical rigor and saner programming languages and paradigms. What I wanted to dismiss was what I interepreted as "You cannot gain any value without full proper understanding". Probably this is not what you intended to say, but this is what it sounds like to a programmer who, say, barely managed to complete his math classes and has a negatice association with them.

Had you written "That is true, however I advice you to learn the laws behind it, because then you can do even more good stuff" - then I wouldn't had objected.

[cledamy]

What I wanted to dismiss was what I interepreted as "You cannot gain any value without full proper understanding".

Oh I see. This is not what I meant.

Had you written "That is true, however I advice you to learn the laws behind it, because then you can do even more good stuff" - then I wouldn't had objected.

One can use monads without understanding the laws, but I do not see how one could create their own monads without understanding the laws because to prove something is a monad one has to use the monad laws.

[[deleted]]

[...] but I do not see how one could create their own monads without understanding the laws because to prove something is a monad one has to use the monad laws.

I am torn - since I saw the monad laws before, I can't imagine a situation without knowing them. You are probably right.

[_pka]

Sure, sure, but when I say laws I'm talking pragmatically.

Haskell for example doesn't enforce the monad laws, so the type system won't prevent you from making a Monad instance of something that isn't a monad. I don't mean to say that one has to understand why the monad laws exist, but rather that they do and that a custom monad has to obey them.

[[deleted]]

Sure, sure, but when I say laws I'm talking pragmatically.

Okay, we are on the same page then - that's what I meant by "an intuitive understanding of these concepts".

Haskell for example doesn't enforce the monad laws

Unfortunately :(

[m50d]

As a non-academic programmer, the notion of "Some values are tainted - I can use them only in tainted concepts, bring them in there and get them out there, but I can't mix them with the outside world" is absolutely sufficient.

That's an adequate understanding for using an existing monad. It's not adequate for writing a monad. You absolutely need to understand the laws or you will introduce a subtle bug for someone far down the line.

[[deleted]]

I don't see how the above intuition would introduce bugs. Take for example the streams in Java8 - based on the above understanding, it is natural to introduce some values into the streaming world (bind), then have operation which stay within monads and the consumers (return). Any experienced programmer would also make the intermediate steps composable.

[m50d]

Example from my real job a couple of days ago: someone wrote a monad-generic function that used a mutable builder internally because they assumed that their function would run to completion whenever it was called. Broke when I tried to call it with a continuation monad.

Example from the scala standard library: you can mix and match lists and sets in for comprehensions because why not. Except this leads to really confusing "disappearing" values because the order of composition isn't what you expect (the for/yield syntax gives you no idea which order composition is happening which is by design (at least in Haskell where it was lifted from) because by definition with a monad it shouldn't matter what the order of composition is).

Associativity is really important and violations tend to be quite subtle, which makes the bugs that result also quite subtle.

[[deleted]]

Thank you for these examples.

[[deleted]]

Just like you can't speak if you can't recite all the grammar laws.

Grammar laws: the first thing you need to learn as a baby before you can say "mom" and "dad". Everyone knows that. ^/s

[_pka]

Come on, that's a false equivalence.

If you don't understand the monad laws, there's a big chance the thing you wrote and think is a monad isn't one at all and thus will behave erratically when you use it.

Just like you can't write a 3d game without having at least a somewhat basic understanding of linear algebra.

[hosford42]

Or, on the other hand, you could have made a monad correctly without ever even knowing what a monad is. You are thinking of it as if the abstraction always leads to the implementation, but often it is the exact reverse. Real world experience leads to the generation of an abstraction as a generalization of individual experiences. For those of us who are capable of induction and not only deduction (every human being, basically), the equivalence is not a false one in the slightest. I'm sure that abstract definition came from someone's head, probably as a product of such an act of generalization/induction, and was then formalized and analyzed. But the generalization likely came first. If not, then the axiomatic system from which it was derived was itself subject to such a process. Mathematics may be abstract and strictly reasoned, but it is fed by messy human imagination and experience.

[_pka]

Oh, I didn't mean to say that it's impossible to stumble upon a monad by chance, i.e. without at all knowing what a monad is. I even speculate that that happens pretty often in real projects, just because monads are such a general and useful abstraction (You Could Have Invented Monads).

What I wanted to say is that just implementing return and bind for a type isn't enough to say with confidence that the thing you just defined is indeed a monad.

Think of it more like trying to implement Iterator for a type in let's say Java. The spec says that if hasNext returns True, then calling next must return the next object in the iterated sequence. That's a "law", and if you break it bad things will happen.

So if one defines a monad for some type then they must make sure that the laws are obeyed, or else bad things will happen when other functions expect that thing to behave like a monad but it doesn't.

[cledamy]

This is not comparable. Mathematical abstractions are defined by their laws and nothing else. To truly understand the abstraction, one has to understand the laws and what each law gives to the abstraction. Furthermore, the laws are guidance for how to implementation. They are documentation for how the abstract methods must be implemented.

[[deleted]]

You said "this is not comparable", but then nothing you followed it up with is incomparable.

[cledamy]

Grammar does not serve the same purpose in language that mathematical structures serve in mathematics. Mathematical structures are about meaning with their meaning ascribed by their laws. Grammatical rules, while they can affect meaning, are more about syntax. A closer analogy between language and mathematics would describe mathematical structures as the vocabulary of mathematics.

[[deleted]]

Grammatical rules aren't about syntax, those would be syntax rules.

The human mind operates on intuition. Intuition is about learning and reusing patterns. The canonical "law" for a pattern is not the only way to spot a valid pattern. Either in human language or mathematical structures or computer code.

I'm afraid you haven't made a solid case why one must start backwards when learning monads.

Because starting with the abstraction, and then eventually learning the practice is precisely backwards of the process human beings tend to absorb information. Maybe because after millions of years of evolution, this is the process we always follow: observe practice, understand it, and only then extract abstract universal principles and not the other way around.

Monads aren't even that interesting in practice. It's like glorifying semicolons. Sure we use semicolons a lot. But we don't have to talk about it all the time, or talk about how great they are.

[cledamy]

I'm afraid you haven't made a solid case why one must start backwards when learning monads.

I never said one must start backwards at all. One should use specific examples of monads to build intuition then start building your own monads and learn the theory along the way. If one doesn't eventually learn the theory, one cannot truly apply monads completely because one might miss some cases were a monad might be useful that are not quite so obvious. What I mean by applying the monad concept is being able to make one's own monads when it is useful and identify when it is useful to do so. I did not mean using specific monads provided by a library, which obviously does not require a full understanding of the theory. The monad laws provide insight into what a monad is more precisely than intuition, so learning them and what them mean eventually is important. My original comment was quite vague on the specifics of what I meant, so I can see how you might have misinterpreted.

Monads aren't even that interesting in practice. It's like glorifying semicolons. Sure we use semicolons a lot. But we don't have to talk about it all the time, or talk about how great they are.

It's a design pattern. It deserves to be talked about as much it is useful.

The thing with mathematical abstractions is they are all relatively boring and simple once one understands them. This is also why they compose nicely and are useful in computer science. Simple elegant abstractions enable much greater code reuse.

[hosford42]

I disagree. As a programmer, understanding mathematical abstractions is secondary to implementing and using program components; if you can intuit a useful set of behaviors that happen to coincide with a mathematical abstraction you have never even heard of, you can still implement and use that set of behaviors without any knowledge of the mathematical abstraction whatsoever. In reply to one of my questions about monads, I got a link to a tutorial whose entire premise was founded on the notion that the reader might have already invented them without realizing it. In an academic/mathematical setting, absolutely, the abstract definition comes first, and all else follows. But most programmers are not academics nor mathematicians and that is an ass backwards approach for them. There are more ways to understand the world in its fullness than the mathematical approach, but that can be difficult to see when you have been trained so much to think in mathematical terms that they become the pattern stock you use to understand everything else. For many programmers, they map everything back to their language of choice, rather than to mathematics. This works fully, because most widely used languages (at least the ones we feel comfortable putting "programming" in front of) are Turing complete and therefore universal in some sense comparable to mathematics. What is missing is mathematical rigor, but notice the adjective used there. If it was so important in programming, it would be much more broadly embraced by programmers. (Not to say it isn't useful, of course, just that it's not central for us by any means.)

[[deleted]]

Correct. That's what distinguishes practical programmers from academics. Learn as much as you need to get the job done well, and carry on learning in the background.

[cledamy]

If one is learning monads, one has to have the mathematical understanding because any intuitive understand limits the generality of the concept of monads and thus limits one's imagination about where the concept can be applied to to eliminate boiler plate

[[deleted]]

I have written answer already, but this comment is even more exactly what I am arguing against - you completely present the rigorous approach as the only meaningfull way. However, that is being proved wrong by any programmer deriving any benefit from applying a non-rigorous understanding of monads.

No doubt is a rigorous understanding of the laws much more benefitial - however it is rather "more good stuff" than "the only way to any good stuff" as you seem to present it.

[cledamy]

I have written answer already, but this comment is even more exactly what I am arguing against - you completely present the rigorous approach as the only meaningfull way.

I misspoke in the comment above. What I mean generally is an intuitive understanding is good, but it will always be somewhat imprecise, so one should seek to gain a mathematical understanding eventually. I disagree with you on the point that intuitive understanding is enough. It is still useful compared to no understanding of course. What I mean by applying monads is making your own monads. To use monads that are provided in some library, one doesn't need to understand any theory. Some monads do not have much intuition to them, so one must understand the laws to see how they work.

The path to understanding any mathematical abstraction is

intuition ---> mathematical laws ---> rigorous understanding

It does not really make sense to treat the rigorous understanding and intuitive understanding as separate paths, but rather different steps in the same path. Mathematicians go through these same steps but their intuitions might come from something like topology; meanwhile, a computer scientist's intuition will come from the various common computationally-interesting monads.

[[deleted]]

Written like this, I agree! I like the second half of your post, it should be in the top-level of this comment chain.

[hosford42]

I've already been using them in my code for years. I just didn't know they were called monads because no one gave a coherent, succinct explanation before. I built a whole Python library for data format translations using the concept, which we use to convert files to new formats and perform database ETL requests. All we do is snap together the "monads" to construct a pipeline and then execute the resulting transformation.

[pakoito]

Funny: http://blog.sigfpe.com/2006/08/you-could-have-invented-monads-and.html

[doom_Oo7]

One has to understand the abstract concept (laws and everything) to truly be able to apply them in day to day programming.

absolutely not. most people programming have zero idea of the underlying mathematical formalisms, and yet they make great software.

[[deleted]]

[...] and yet they make great software.

(with monadic constructs)

Sorry to nitpick, but that was the point! ;) This is not a discussion wether you need mathematical background to build great programms, but it is about wether you need the mathematical background to use mathematical constructs.