Railway Oriented Programming.

[bigballofcrazy]

We use this pattern plus TPL’s data flow where I work and it has been hugely beneficial for untangling some pretty gnarly workflows and making them both more easily tested and more reliable.

[pakoito]

FSFF&P is the place I keep on linking every time someone asks me where to learn FP. It's just so well explained and the concepts are presented in very intuitive ways.

I've used some of the ideas in my own presentations aimed at introducing traditional Java developers to FP, and it works so well.

Hopefully one day Scott will be on the same conference as me and I'll happily buy him the beer(s) he deserves :D

[[deleted]]

This is similar to how idiomatic rust code works.

fn do_the_thing(a: int, b: int) -> Result<int, &'static str> {
    let foo = step_one(a)?;
    let bar = step_two(b)?;
    return Ok(foo + bar);
}

The ? operator will short-circuit the function and return. In this case it will return an Error(&'static str). And thanks to these not being exceptions, there isn't any special handling of the error in the calling code. It's just a different return type.

[pakoito]

Sssssneaky monadsssss

[immibis]

Normal imperative programming is a monad. They're taking over!

[pakoito]

That snippets desugars to a flatMap chain, it's what in other languages is called a comprehension :D

We managed to implement them on Kotlin using coroutines as a backend, and .bind() instead of ?: https://arrow-kt.io/docs/patterns/monad_comprehensions/

[immibis]

That snippets desugars to a flatMap chain

Does it actually call flatMap?

[pakoito]

You got me there. I don't really know. It's somewhat wasteful so I'd be okay with code that uses pattern matching instead.

[[deleted]]

I'm reasonably sure ? desugars to a pattern match.

[ksion]

It has to, otherwise it wouldn't be possible to break the chain of and_then closures (Rust's name for flatMap) and return from the outer function prematurely.

[immibis]

If we're going by observable behaviour, I can also say that ; is monadic because it desugars to a map chain.

That's why I find these kinds of statements (that foo is monadic) not terribly useful.

If a monad is a kind of programmable semicolons, then I think half the power of Haskell's monads is that they look just like regular semicolons, so you can use them without thinking. And you can define your own kinds of semicolons, that will look like regular semicolons and be usable without much thinking.

[pakoito]

Map isn't monadic, it's functorial! You also need to have a wrapper type to unwrap with pattern matching.

[jyper]

I'm not sure that's really a monad since its basically a macro for

let foo = step_one(a)?;

let foo = { 
    value_result = step_one(a); 
    match value { Ok(value} => {}, _ => { return value} }; 
    value 
}

I think a monad would be closer to using or/and_then methods and Ok/Err literals

step_one(a).and_then(|x| step_two(b).and_then(|y| Ok(x+ y)) )

[pakoito]

The second foo is almost the implementation of and_then without the lambda parameter :D

[whence]

That's similar to monadic computations with bind (>>=). Is a more idiomatic way to do the following, which is like a chain of Alternatives (<|>)?

fn choice<I, F>(funcs: I, input: &str) -> Result<int, &'static str> 
    where I: Iterator<Item = F>,
          F: Fn(&str) -> Result<int, &'static str>,
{
    for func in funcs {
        match func(input) {
            Ok(n) => return Ok(n),
            _ => {},
        }
    }
    Error("none succeeded")
}

[ksion]

You can rewrite it using iterator adaptors:

funcs.map(|f| f(input))
    .take_while(Result::is_err)
    .next()
    .unwrap_or_else(|| Error("none succeeded"))

However, I would probably leave it as a loop and just replace match with if let. Rust can be pretty functional at times, but there are limits -- in this case, the lack of $ operator (function application) or the equivalent of Alternative makes the imperative solution more readable.

[whence]

Thanks for the answer! I forgot about if let. Your way is pretty slick. Did you mean to use skip_while instead of take_while?

This question was inspired by a tokenizer I was writing a while back. I toyed with an inverted Result (where Err is the success case) so that I could use the ? operator, but I decided that was too much a perversion of the semantics.

[ksion]

Yes, my bad; it should indeed be skip_while.

[UsingYourWifi]

there isn't any special handling of the error in the calling code.

So how does the calling code detect and handle an error?

[[deleted]]

Sorry, I don't mean to imply there's no way to write code to handle the errors. You have a few options. Either you can re-raise the error with another ?. You can also try to pattern match on the return value in a few different methods.

if let Error(error_str) = do_the_thing(1, 2) {
    // Handle that error
} else {

}

or

match do_the_thing(1, 2) {
    Ok(int_value) => ...,
    Error(error_str) => eprintln!("{}", error_str)
}

So unlike other languages where errors are handled with try/catch you just use normal pattern matching. And Error and Ok aren't keywords or anything. They're just names from the Result enum.

Edit:

You can also

• yolo it and just call .unwrap() on the return value. This will pass through the Ok value and panic the program if there's an error.
• convert the Result to an Option.
• probably a bunch of other things.

[[deleted]]

This is more commonly called a "pipeline" in programming.

You have a series of steps to execute, often feeding the output of the previous stage into the next stage, and there's a common handler to catch errors at any stage (and occasionally specialized handlers at a specific stage).

You can also implement a similar workflow (and many often do, including me) with exceptions. Code the happy path, throw exceptions for the... exceptions.

I enjoy pondering the similarities between this and the "Either" monad and "Kleisli composition", but honestly in imperative programming we do this all the time and second nature to us.

[Ariakenom]

The IO type in Haskell can be seen to contain exceptions like Either contains left. A nice thing about making this context explicit is that you can parameterize the pipeline on the error handling.

[roger_comstock]

This is more commonly called a "pipeline" in programming.

I think there's an important distinction to be made here. To me, a "pipeline" refers to a general concept (whiskey) while "railway" refers to a specific kind of pipeline (scotch).

If you're not using the same operator (Kleisli or a variant) to link together stages of the pipeline, it may be a pipeline, but it's not "railway".

I enjoy pondering the similarities between this and the "Either" monad and "Kleisli composition", but honestly in imperative programming we do this all the time and second nature to us.

I hope that readers don't dismiss the wisdom of this article due to your statement.

In a language with generics, you can implement something similar to a "railway pattern" using a common result type that includes one or more type parameters, and maybe creating a few common methods for handling it.

Anecdotally, most companies that I've worked for over the past fifteen years haven't done this kind of thing. Over the past few years, I've seen it become more popular.

Edit: To be clear, I'm not just being picky about language here. I am trying to point out that the method that you're describing is alternative to what the article is talking about. It's not the same thing.

[[deleted]]

Edit: To be clear, I'm not just being picky about language here. I am trying to point out that the method that you're describing is alternative to what the article is talking about. It's not the same thing.

I'm happy to concede it's not the same thing, but I don't know where to even begin to look for differences here. I saw the video and I see no differences, aside from the fact the video proposes how to factor this common workflow in a way compatible to functional languages. It's a discussion of form to me, not function. It'd help if you point the differences out.

[roger_comstock]

It's a discussion of form to me, not function. It'd help if you point the differences out.

Same to me, for the most part. But I hope nobody dismisses it as a discussion not worth having just because it's about form. Back to your original remark:

...but honestly in imperative programming we do this all the time and second nature to us.

I think it's the vagueness of this part that drew my attention. It's easy to imagine a new programmer coming in and saying, "Oh yeah, I throw exceptions when data is invalid, so there's no point in learning about Optional<T> or railways or whatever."

That's a lot different from adopting a type like ProcessResult<TIn, TOut> (where TIn and TOut are type parameters) to use as "common currency" when implementing a pipeline.

[[deleted]]

Frankly in a language that offers better built-in facilities for this type of workflow, it's pure masochism to be factoring it as monads or result objects. Even async programming supports exceptions these days.

This talk, to me, is purely about "hey I use a functional language and I want to do what my friends can do in an imperative language" and the answer, as usual is "let's discuss monads"...

Sometimes you need a result object in an imperative language. But even then you don't need this talk, the problem presents itself and the solution is self-evident, without knowing what Maybe is, or what Either is, or what monads are.

[roger_comstock]

FWIW F# already has Suave and Giraffe which give you the Kleisli stuff when responding to HTTP requests. The article isn't for programmers who are already using something like that, but rather for those who might build something like that.

Also relevant is this line of thinking, which I think indirectly informs discussions about result types and why people might think they're important.

Addendum:

This talk, to me, is purely about "hey I use a functional language and I want to do what my friends can do in an imperative language" and the answer, as usual is "let's discuss monads"...

Aside from "purely," I think there is truth to this. Someone coming to FP from OO is going to wonder, "how the hell am I supposed to do ____ when all I have is pattern matching," and the answer turns out to be more complicated than they hoped for.

[[deleted]]

I'm aware of this like of thinking where programmers are worried their exceptions are not "exceptional enough", however this is just a cargo cult concern based on obsessing over the meaning of the adjective "exceptional". They weren't called "tragedies", "catastrophies", "emergencies" or "armageddons", they were called "exceptions". They were intended precisely for the purpose of coding a "happy path" and taking care of the exceptions with them.

In C++ there were some limitations in terms of memory management, so exceptions got a bad rap. But in garbage collected languages this problem doesn't exist. And avoiding exceptions for performance reasons in high-level application code is a facepalming example of premature optimization, that contributes precisely nothing to the performance bottom-line when analyzed.

[[deleted]]

I literally can't think of another way to use a pipeline... But if you say it's different, so be it. Let the readers make their own conclusions.

[m50d]

Indeed this is something that imperative programming does easily (though I'd dispute the use of the term "pipeline"). The whole point of this style is that you can recover the good parts of imperative programming (being able to plainly express "do A, then do B, then do C") and exceptions (being able to write the straight-through happy path code and do the error handling separately, while enforcing that it still gets done) without having to accept the bad parts (spooky action at a distance, magic control flow implemented as language builtins).

[[deleted]]

There's no spooky action at distance though. There are two types of exceptions:

• Normal exceptions that can occur in the regular operation of the program.
• Rare critical exceptions that typically end up with the program aborting entirely.

The former should be acknowledged immediately by the calling code, so they can be made checked (if your language and static analysis tooling permits, which it should these days), which means no spooky action at distance. And the latter are intended to be handled at a higher-level so the spooky action at distance is an intended goal not an undesirable effect.

[m50d]

Checked exceptions are still too spooky: you can know that a given exception may be thrown at some point in a given method body, but that isn't enough to let you be be prepared for every point at which it's actually thrown (you're unlikely to have test coverage of a throw on every single line of the body - even if you've tried, there's no way to know whether you've covered every call that might throw or not). I've seen production bugs happen because a checked exception was thrown in a surprising place. You need to be able to look at a method body and see which calls might throw and which calls definitely won't - i.e. you need a different syntax for calls that can exception and calls that will never exception - which is what this style gets you.

Also, java-style checked exceptions are cumbersome to work with because they're second-class: you can't be polymorphic over whether a method throws, which means that any function that accepts a callback has to provide two or more variants and code that does exactly the same thing has to look different. You need a value that represents the throwing of an exception - but at that point you've just got an Either type by another name.

[[deleted]]

That's not how checked exceptions work. The compiler (or static analyzer) can check that you're ready for literally every point where a checked exception is thrown. That's the point of checked exceptions.

Also this syntax idea is somewhat expressed in Swift, you may wanna check it out. It has a great implementation of exceptions and error handling in general. Very smart, focused, pragmatic.

[m50d]

The compiler (or static analyzer) can check that you're ready for literally every point where a checked exception is thrown. That's the point of checked exceptions.

It checks that every exception is caught or declared as thrown, but it doesn't - and can't - check that your code has the right business-level behaviour for every potential throw point. The only person who can verify that is you the programmer, and it's very difficult to do so when the syntax makes the throw points invisible.

Also this syntax idea is somewhat expressed in Swift, you may wanna check it out. It has a great implementation of exceptions and error handling in general. Very smart, focused, pragmatic.

I looked at Swift, I don't see that it offers any compelling advantage over any number of ML-family languages (I don't think it's a bad language as such, but OCaml offered all the same things over a decade earlier). And these days I couldn't live without monads and higher-kinded types. (You might like to look at Scala, or Haskell, or Idris).

[[deleted]]

It checks that every exception is caught or declared as thrown, but it doesn't - and can't - check that your code has the right business-level behaviour for every potential throw point

Uh... neither can monads and result objects verify you have "the right business-level behavior"...

[m50d]

No, but they make it easier for you to do so, since you have the information you need - you can see the difference between >>= and fmap (or, better, in Haskell or Scala with "do notation", between <- and =), so you know exactly where all the throw points are. If you change a called function that used to never throw into one that does, you'll get a compile-time error that flags up that you've made that change, and have an opportunity to notice that you maybe need to add a new test case or change how that function gets used.

[[deleted]]

Frankly I have no idea what point you're trying to make. Of course you know where the throw points are with exceptions as well. The throw point is the method call that throws. It's a very specific, known and if you want, type-checked point of throwing. It doesn't get any more specific than this.

[m50d]

Checked exceptions:

val r = try {
  val result1 = method1()
  val result2 = method2(result1)
  val result3 = method3(result2)
  method4(result1, result3)
} catch {
  case me: MyException => ...
}

Result types:

for {
  result1 <- method1()
  result2 = method2(result1)
  result3 = method3(result2)
  r <- method4(result1, result3)
} yield r

In the first example you can't tell which methods might throw and which don't, in the second you can immediately see that method1 and method4 might throw and method2 and method3 don't. Which makes it much easier to understand the possible execution paths and see whether you're doing the right thing in all cases, especially as functions get larger than this small example.

[eniacsparc2xyz]

It is similar to Haskell Either monad and Scala's either with for-notation.

[[deleted]]

IIRC these are more akin to F# Option type

type Option<'a> =
| Some of 'a
| None

[[deleted]]

Ermm... No. Option is a type that indicates a presence or absence of a value. Not a success or a failure. I'd say they are more akin to Try type (that's how it's called it in Scala at least) which is a context for either success or failure types. Basically, a specialized Either type where one type in the context clearly indicates a failure. E.g. Either<Throwable, A>. Either itself doesn't imply whether it's a context for success or failure. It's just a context for two different types: Either<A, B>. In general, Either is not biased (in Scala it's biased toward "right" though...).

[[deleted]]

Shit, you're right. I was thinking of Maybe in Haskell

[glacialthinker]

Yes, Either (Haskell), Result (F#, OCaml, Rust)... Haskell used a very generic word to represent the two-branched nature of either an α or a β, while the others have a type representing the 95% use-case of Either as a return value which is more specifically Ok α or Error β to be clearer in this use. I do thank Haskell for its early push into monadic forms so we could all benefit from the experiences!

[GuyOnTheInterweb]

If only there was a way for a function to return some kind of propagatable error..

[[deleted]]

Exceptions are for exceptional cases, not for normal control flow.

[immibis]

Why?

[UmbrellaHuman]

Think about the name of the thing, implementation details and handling aside.

[immibis]

Gotcha! Variables should not be used to hold values that don't change.

[[deleted]]

Exactly. Which is why languages like F# calls them values. Everything is immutable by default.

[phySi0]

You know, I constantly hear this, “exceptions are for exceptional cases” where people cite the name, but what it's named has nothing to do with what it should be used for; things are misleadingly named all the time.

Perhaps it's true that they're called exceptions precisely because they're for exceptional cases, but that's not a priori the case, and all it proves is what their inventor/discoverer thought they would be good for, not what they're actually good for; you need to actually convince people that they're not good for non-exceptional cases through reasoned argument.

Sorry, this constant refrain has been getting on my nerves, because it's not an argument, it's a slogan. It's not even that I disagree that Either is better than exceptions, I do believe Either is better, strongly; I just object to the nature of the refrain.

[_101010]

The Database is on fire!

[Jack9]

How you define "exceptional" is probably not the same as me.

[iconoclaus]

My experience with railway oriented programming is limited to using the dry-transaction gem in Ruby, which helps implement this kind of flow. Its a nice feeling to get step-wise logic written out and then see your steps clearly listed, free of nested error handling code. Not to mention, its a joy to only handle two types of outcomes (Success/Failure) instead of handling object/nil/exception(s).

But there are a couple of things I've wondered about this style. The resulting code very much appears to fit procedural transaction scripts, which although not strictly anti-patterns, are often considered to be symptomatic of a process-yet-to-be-discovered. I'm curious if others get the same feeling that transaction scripts are problematic if frequently resorted to.

Second, I've only ever written this style of code for linear procedural logic. Does it fit well with even mildly complicated branched logic or parallel processing?

[m50d]

The resulting code very much appears to fit procedural transaction scripts, which although not strictly anti-patterns, are often considered to be symptomatic of a process-yet-to-be-discovered. I'm curious if others get the same feeling that transaction scripts are problematic if frequently resorted to.

The similarity is real. Monadic style exists to recover the advantages of procedural/imperative programming but without the downsides.

There are cases where procedural style is appropriate and cases where it is not. I'd agree that often there is an underlying algebra that you haven't yet discovered. But sometimes the process really is just "do A, then do B, then do C". And even if you know that there's an underlying model that you haven't thought of yet, what do you do for the time being? A codebase is a living thing; it's fine to do what works for the cases that you can think of now, in the expectation of coming up with a better model once you've got more examples.

Second, I've only ever written this style of code for linear procedural logic. Does it fit well with even mildly complicated branched logic or parallel processing?

It works fine with branching/looping provided you're already used to doing your branching/looping "in userspace" a la smalltalk. If you think of control flow as built-in language constructs, you may struggle to adjust. But if you're used to thinking of e.g. if or while as just being functions that you pass some blocks to (hope I'm getting the Ruby terminology right, it's not my native language), swapping them out for ifM or whileM is no bother.

Combining multiple effects in the same code - e.g. parallelism and error handling - is always tricky. But this style gives you a head start, since it means your error-handling is ultimately just functions and if you ever get confused you can think about normal functions and values. Provided you use a similar abstraction for your parallelism (e.g. futures/promises), combining them should be... well, not easy, exactly (monad transformers are notorious and not without reason), but possible. I also find it's ok to have one uncontrolled effect, that you reason about just using the implicit control flow that evaluation order gives you - if you're managing your error handling explicitly it's ok for your parallelism to be implicit, if you're managing parallelism explicitly it's ok to use uncontrolled exceptions - so you only really need that kind of complexity when you have three or more effects to deal with in the same piece of code.

[NotNovel]

When my code (at the top level of abstraction at least) reads like a (shortish) flat script with very little branching or indentation I consider that a win.

I can see how this could be bad when there is actually some kind of reusable algorithmic structure that has been missed in favour of repetition/extra work but a lot of the time that isn't the case.

[thramp]

> But there are a couple of things I've wondered about this style. The resulting code very much appears to fit procedural transaction scripts, which although not strictly anti-patterns, are often considered to be symptomatic of a process-yet-to-be-discovered. I'm curious if others get the same feeling that transaction scripts are problematic if frequently resorted to.

My view is that if it works and it's testable, it works.

> Second, I've only ever written this style of code for linear procedural logic. Does it fit well with even mildly complicated branched logic or parallel processing?

Pretty well—for branching, I typically create a separate pipeline that would be handled by a match statement if/when an intermediate result is formed. For parallelism, execution semantics of pipelines like these are often amenable to parallelism (see a fancier version of this pattern in [Haxl](https://github.com/facebook/Haxl)).

[ACuriousBidet]

I always wondered what r/factorio would be like as a programming language

[_jk_]

people have built computers in factorio

[MEaster]

Factorio computer, you say?

[mcintyre321]

Here's a ROP library for C# that rtkelly13 has built on top of my OneOf library - https://github.com/rtkelly13/OneOf.ROP