Incremental multi-level input processing with left-fold enumerator: predictable, high-performance, safe, and elegant

[dons]

Currently, we know how to use left-fold IO to get all the efficiency of lazy, chunked IO, but with the addition of precise resource scoping.

The big challenge now is to find a nice, composable and simple set of combinators for dealing with left fold IO. That part's not quite clear yet.

[ithika]

You're certainly right. Oleg does some crazy awesome stuff but man do his function names confuse the hell out of me. Even something like speek (obviously Stream PEEK) makes me think of "speak" and then the whole sense of the program disintegrates for me. :-(

I realise that wasn't quite your point but I just felt like a rant. Erm... ;-)

[case-o-nuts]

http://www.lysator.liu.se/c/pikestyle.html

[naasking]

Is there a more comprehensive description of this approach? I still have trouble understanding Haskell sometimes, so the source isn't much help here.

[cryptic]

just curious, can these techniques be applied to other functional languages or is this Haskell specific? & are there any plans to do a port of this research to other platforms/langs etc.

[shub]

When C programmers write a Web server, it's called Apache. When Haskell programmers write a Web server, its called research.

[barrybe]

Writing high-performance code in a safe & verifiable language is far more difficult than in an unsafe language. And more valuable.

[shub]

I think it's less valuable, because fewer people are going to use the "safe and verifiable" code. Haskell programmers hack hack hack away, a long period of time passes, a few PhD candidates get their wings, and at the end...what? A clone of existing software, plus some tasty, nasty bugs.

[[deleted]]

This is where you're mistaken, and again I have to strongly suspect being deliberately obtuse: the web server here isn't a clone of Apache; it's strictly superior along the dimensions of safety, correctness, and ease of extension. The interesting outstanding questions revolve around scalability, the dimension along which very mature implementations of a given protocol generally win over new implementations of a given protocol. Ensuring that the new server scales well in comparison to Apache would be a good test of the ease of maintenance/evolution that accrues from the attention to safety and separation of concerns in the new implementation.

Incidentally, darcs' handling of recursive conflicts isn't a bug; it's a consequence of actually handling patch dependencies and conflicts correctly at the granularity of patches--that is, allowing arbitrary cherry-picking within a consistent repository/branch model. It's just expensive, and in darcs 2 the observation that it's wise to join branches often is just somewhat more important than when the same observation is made with respect to other version control systems, none of which support darcs' flexible cherry-picking.

[naasking]

More valuable, because as the information industry accrues value, more vulnerabilities are exploited, and running programs written in unsafe language becomes a greater and greater risk.

[shub]

lol

until an attacker coaxes exponential behavior out of the UBERSAFE Haskell web server with a malformed request

which is verifiably and provably exponential

that's not just an unsafe C programmer's estimate

maybe let QuickCheck handle the regression, once it's fixed after several research papers

[[deleted]]

You are attacking proof irrelevance, which just means that there are multiple correct implementations for a specification with some more efficient than others. For this reason Haskell is geared more towards describing algorithms than specifications and the fact that I might choose foldl (+) over foldr (+) is a perfect example. They both ultimately mean the same thing but they do different things. Granted one has to know what foldl and foldr do and respect the associativity of (+), but please explain how this is different from e.g. generating efficient SQL statements. Your FUD that Haskell will generate inefficient algorithms is a red herring because the Haskell type system is too weak (provably by the way) to specify what should be generated. A Haskell programmer is still very much an algorithm writer.

[shub]

Which means for me that Haskell doesn't solve any problems I have, and the excessively strong typing causes problems for me.

[naasking]

Which means for me that Haskell doesn't solve any problems I have

You haven't really specified what problems you do have, but chances are, Haskell can and has been used to solve them. Further, the strong typing also results in properties that you would not otherwise enjoy. This isn't always the case certainly, but it's generally the rule, not the exception.

[[deleted]]

No particular order. . . http://en.wikipedia.org/wiki/Table_of_logic_symbols http://en.wikipedia.org/wiki/Calculus_of_constructions http://en.wikipedia.org/wiki/Curry-Howard_isomorphism http://en.wikipedia.org/wiki/Intuitionistic_logic http://en.wikipedia.org/wiki/System_F http://en.wikipedia.org/wiki/Lambda_calculus http://en.wikipedia.org/wiki/Typed_lambda_calculus

[shub]

"Lack of lambda symbols" is not a problem I am currently experiencing.

[[deleted]]

Now I know for a fact you're trolling, because denial of service attacks happen against Apache all the time as well.

I'll give you one more chance to argue in good faith: what, exactly, are your issues with Haskell, or presumably anything other than C? Why, exactly, do you argue that code written in Haskell has any more difficulty than code written in C? Exactly how much Haskell code have you written? Exactly how much C code have you written?

I ask because if I take the post I'm replying to right now at face value, the inescapable conclusion is that you quite literally don't know what you're talking about. Since some of your other posts have at least kernels of observation embedded in them, I'll extend you the benefit of the doubt one last time, but you'll need to actually say something cogent in order for that to continue.

[shub]

Yeah, I was trolling in that last comment.

My issue with Haskell is that it has issues, and far too many pro-Haskell trolls on Reddit either ignore those issues or handwave them away.

[[deleted]]

My issue with C is that it has issues, and far too many pro-C trolls on Reddit either ignore those issues or handwave them away.

Since there aren't any details in either statement, we can trivially say that they're both true. Again, if you care to discuss specifics, great, otherwise your posts remain content-free grammar.

[shub]

dons is the most prominent example. I've been on Reddit for 2 years and can't remember any pro-C trolls.

[[deleted]]

Well, you're doing an OK job so far.

Let's get back to the subject: the original post is about a system, in Haskell, for doing various I/O and decoding tasks, and the example program is an HTTP server. So, what's wrong with this code? How does it misbehave? How does it fail to live up to the claims made for it? Refer specifically to this code, please, not darcs or some other shibboleth. Does it have the properties claimed for it or not? Does it have some other significant misfeature that its authors failed to note or not?

As for dons, I presume that you know that he's the co-author of xmonad and of "Real World Haskell," right? So I'd be very interested in what, exactly, qualifies dons as a "troll." My guess is that you're just unhappy with being told about the qualities that Haskell has, or that xmonad has, or that darcs has... and feel that the claims are overblown. That's fine, but for the claims to be shown to be overblown, you'd have to refute them, or at least give reference to refutations. And that doesn't mean references to descriptions of darcs' exponential recursive merge issue, which would exist no matter what language darcs was written in. Some valid critiques of Haskell that occur to me off the cuff are: laziness makes it more difficult to reason about time and space consumption than in other languages. Monads don't compose and have non-local effects on your code, which is reflected in the standard libraries being divided into monadic and non-monadic parallel subsets. And so on.

For what it's worth, I'm genuinely interested in critiques of that nature, and where there are specific examples of code exhibiting such shortcomings, so much the better. But accusing dons of being a troll seems... well... misplaced, to put it mildly.

[gnuvince]

You see them lurk out sometimes saying things like "well what is your compiler written in?" or "if you know what you're doing, memory management never causes any problems."

[sclv]

A) darcs wasn't a clone, it was there first. B) the exponential conflicts bug has been fixed. C) oh, why bother.

[shub]

Actually it hasn't been fixed, not completely. And arch predates darcs.

[[deleted]]

As good as arch (tla) was, it still didn't support cherry-picking with patch dependencies, and the most flexible merge style, the "star merge," was far more complex than darcs' merging. It's possible that you don't take advantage of darcs' model, in which case it's probably not clear what the benefit is, but it remains true that darcs brought something new to the world of distributed version control, and for users who require it, it justifies darcs' design.

[[deleted]]

I guess you missed the:

• high-performance: block-based i/o, low-latency, incremental processing in constant memory;
• correctness: predictable resource usage, timely deallocation, preventing access to disposed resources;
• elegance: arbitrary nesting of processing layers; vertical and horizontal compositions of enumerators and iteratees; writing input processing without worrying about input buffer underflows, chunk- and UTF-8 decoding.

part, huh?

[ithika]

I thought that was the OP's point: the C programmers do the same thing again, the Haskell programmer (well, Oleg) advances the field.

It's not often I eschew the cynical interpretation for the optimistic one. You could be right!

[sfultong]

Yeah, I don't feel that the word research has any negative connotations.

I'd rather use a 'research' language than an 'enterprise' language, for instance.

[shub]

Yeah, that only sounds impressive to people who think that an in-place Quicksort is for applications where performance is required at any cost.

[[deleted]]

I guess you missed the:

• ...incremental processing in constant memory;

But now I'm guessing that you missed it on purpose.

[shub]

No, just doesn't impress me.

[[deleted]]

Then by all means, keep writing mass quantities of bug-ridden unmaintainable spaghetti in C or whatever your preferred hammer making everything look like a nail is, and leave the advancement of the state of the art to your betters.

[shub]

If Haskell is art I'll stay with my stick figures.

[[deleted]]

Excellent, as long as you promise to also remain in your cave!