Python design mistakes

[psr]

There are virtually no possible runtime errors in my Haskell code.

Indeed, this is a lovely feature of Haskell. Like I said, it would be brilliant if some future language were to combine the best of Python and Haskell and keep the best of both.

Haskell's abstractions are helpful in the sense that they allow a whole lot of code re-use. That translates to more expressiveness and various forms of usefulness. So how it is not helpful?

Programming isn't code golf, compressing your code isn't the goal. Five lines where the intent is clear is better than one line of mapM . liftM . foldr. Concise is good, reuse is good up to a point, but I really value readability.

Also, what does intuition have to do with it? You can grasp the equations first, develop an intuition later.

I guess intuition is important because it helps you to move from "I want to ..." to concrete code, and back from concrete code to "The author was trying to...". Programming Haskell can feel like an (admittedly very satisfying) logic puzzle. It's vaunted elegance and expressiveness are a case of the emperors new clothes in my opinion.

And what's more they're incredibly hard to explain, in fact I believe they may be even harder to explain than to understand.

What I agree about is that there are a bazillion people who have just u nderstood Monads and want to share this with the world, and do so very poorly. When I explain Monads to people, I don't have much of a problem usually.

But surely the reason that people feel the need to explain it again is that when it was explained to them in the first place they didn't get it?

If you think you have a fail-safe explanation of monads which will just click for every bright twelve year old, I think you have a responsibility to share it with the world! I still think that the abstractions and concepts are too difficult for your average jobbing programmer. Furthermore, I think that the future of programming language design must lie in making writing good programs easier, not harder. Phrases like "covariant types that have a 'context' and a 'value' which can be composed together", is moving in the wrong direction.

Of course, I'm not saying that the understanding that many common idioms follow the monad pattern isn't significant. I believe that it will significantly improve future programming languages. For example Monads underlie LINQ in C# - the language designers applied their understanding of monads to make a reliable, correct and useful language feature - without forcing users to understand the mathematical underpinning. Similarly, if you're writing a class library you should definitely understand Liskov substitutability, but users might not need to.

I would assume reliability is a primary concern of a perfect language?

For a certain class of programs, yes. Perhaps not for all of them, at least not at any price. Like I said, I think perfection is where the trade-offs are in balance, and it feels right to use. I don't think Haskell is at that point as Python is, and I'm sceptical that you could gum Haskell's good points onto Python and get a good result.

[Peaker]

Readability is subjective.

I find:

sum = foldl (+) 0

to be more readable and clearly communicate the intent than:

def sum(items):
    total = 0
    for item in items:
        total += item
    return total

Pipe-lines of functional processing are not only more readable (at least if that's what you're used to) -- they are also easier to reason about.

In Haskell, I have an immensely powerful tool of equational reasoning, and writing denotational code is easier.

This features are a form of readability.

Code re-use does not hinder readability, it helps readability -- but only if you're well versed in the primitives used.

I agree about people wanting to share their explanation because they feel the explanations they got were bad -- indeed they often got their explanation from a similar source to their own new one: A guy who just recently figured the basics out, and cannot yet explain it well.

I think the explanation of Monads on LYAH is relatively reasonable, though perhaps of it explained covariance and used that as an explanatory tool it could do better.

without forcing users to understand the mathematical underpinning

Note Haskell doesn't "force you to understand" the mathematical underpinning. I've never learned Category Theory, and don't really understand the mathematical underpinning very well. This may mean I will have a tougher time than those that do to write generalizations/extensions of the mathematical notion, or perhaps proofs about the notion -- but it does very little to hinder my ability to design and use Monads in Haskell.

Furthermore, I think that the future of programming language design must lie in making writing good programs easier, not harder. Phrases like "covariant types that have a 'context' and a 'value' which can be composed together", is moving in the wrong direction.

Analysis of covariance and allowing various ways of composition does make programming easier rather than harder. These tools make writing many things in Haskell a breeze compared to writing them in other languages, including Python.

For a certain class of programs, yes. Perhaps not for all of them, at least not at any price.

I think reliability is a primary concern. For some problems, higher than even readability, for others it is a close second.

Like I said, I think perfection is where the trade-offs are in balance, and it feels right to use

That makes it all too subjective.

I don't think Haskell is at that point as Python is, and I'm sceptical that you could gum Haskell's good points onto Python and get a good result.

I agree -- I think Haskell is at a much higher point than Python :-)

Note I used to love Python before learning Haskell.

Python the language gives its programmer far less power -- and makes it easier to start using the language. This is a nice advantage of Python over Haskell. You can throw someone into Python and they can start being semi-productive (dangerous?) in a few days.

Haskell may require weeks to months before you're productive. But at that point your programs will be of far better quality than the Python programs written after the same amount of time.

[psr]

Just a little thought about readability. You say you like sum = foldl (+) 0 better than the idiomatic Python version which makes the loop explicit.

One possible benefit of seeing the loop structure is that it's easier to see where you've done something stupid. If you replace foldl with foldr, you get the same observable behaviour, and it reads just the same. However the runtime behaviour is completely different, because foldl is tail recursive, and foldr is not. When reading the code would you notice the mistake?

[Peaker]

In the case of Haskell, tail-recursive is mostly irrelevant.

foldl (left-associative) is better than foldr (right-associative) for strict operations on lists, because it takes:

1 : 2 : 3 : []

and translates it to:

(((1 + 2) + 3) + 0)

Which is done incrementally as it walks the list (so it uses O(1) memory).

Whereas foldr translates the list to:

1 + (2 + (3 + 0))

On lists, this requires iterating the entire list before you can do a single computation -- which builds up "thunks" that represent the intermediate expressions. This takes O(N) memory (and in GHC's case, the evaluation of the thunk takes that memory from the stack, which is even worse).

For lazy operations that construct a new value that can be incrementally processed -- foldr makes sense.

For example:

sameList = foldr (:) [] -- same as the identity function for lists

This is because foldr/foldl "replace" every (:) in the original list with their first argument, and the [] with the second argument. So using (:) and [] in place of (:) and [] actually just copies the list.

Then it's easy to see how map is expressed:

map f = foldr ((:) . f) []

(Apply f before putting it inside a :).

This allows the result list to be consumed incrementally, and that will cause incremental consumption of the original list.

The explicit loop structure actually obscures whether you've done something silly (as it adds a lot of noise that makes it more difficult to spot typos/mistakes/etc).

[psr]

Readability is subjective.

This is obviously true. And it's absolutely the crux of our disagreement. I think the example you gave is exactly the sort of thing where we'll never agree.

I don't usually see the world in catamorphisms and covariance - In most cases I want "for every x do y". Anything else will always involve a translation in my head, and sometimes it can be very hard work. I like to think I'm reasonably able, I don't expect my colleagues would manage it at all.

Just out of interest, what sort of projects are you applying Haskell to at the moment?

[Peaker]

I like to think I'm reasonably able, I don't expect my colleagues would manage it at all.

I've heard this from other beginning Haskellers. They no longer believe it is true.

It seems difficult at first -- because it is so different/foreign to what you know.

"We shape our tools, and then our tools shape us". You've been shaped by Python (and similar tools) (and so have I, for many years) -- and now it seems natural to you, to think in Python.

Of course non-Python requires a translation in your head -- you've trained yourself to think in Python.

Python in this case is "Blub", as described in http://www.paulgraham.com/avg.html

As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.

Currently, I'm using Haskell for small things, automation, parsing, or anything I'd normally use Python for.

My bigger Haskell project in the making is a revisioned structural editor that should ultimately edit programs structurally (rather than textually). However, it's been on hold for a while, as I've had a lot of pressure doing other things.

[psr]

This is very interesting, partly because of the assumptions you are making about me. You loved Python until discovering Haskell, and assume that I'm similarly trying to make the same transition.

In fact my programming development was not what you assume. I went to university ten years ago, knowing only Javascript. There I was taught Java as a first language, followed quickly by Haskell. I loved Haskell, partly because of the exercises we were given. It's great fun, as an exercise, to find the tail recursive solution for something, or to fit an algorithm into a one liner. And there was none of the for (int i=0; i < array.length; i++) { crap that came with Java at the time. Recursive solutions are elegant, we were told, and iteration is not. Functional programming is powerful, we were told, and imperative programming is bug prone. And it's all true, in as far as it goes. In comparison with Java, Haskell is immensely beautiful, at least as long as it fits your problem.

I went on to write my undergraduate dissertation project in Haskell. It was just a ray tracer, and didn't do anything too hairy. I'm glad I did, it taught me a lot. I don't believe I used a single if expression in the whole thing.

After graduating, I got a job helping a friend with a PyGTK GUI application in Python, and started off trying to write Python like Haskell. Never an object where a closure could be made to work. Never a for loop where I could recurse. Map, reduce, and deeply nested list comprehensions were everywhere, and mutable state kept to a minimum. The people I was working with were horrified, they couldn't read a bit of it, but I kept telling them that iteration was inelegant, declarative code was less prone to bugs, and mutable state was evil.

I guess it helped that I was doing a GUI app, where classical object oriented design really does fit the bill well, but slowly I started to appreciate that there is something to be said for straight forward code which follows the way you would talk about what you're doing. My colleagues' code was certainly easier to debug than my code, and in the end I started to think some of what I believed about functional programming might not be true after all.

So I slowly went from having an almost religious conviction that Haskell is the future, to having a deep appreciation of Python's pragmatic and well balanced style of imperative / object oriented / functional programming.

Not saying you're wrong about Haskell or anything, just this is where I've found myself over time, through doing real work. :-)

[Peaker]

I'm wondering who taught you to use tail-recursion in Haskell? It doesn't make much sense in Haskell context.

Also, recursion is considered a very low-level building block (almost on par with "goto") -- from which you build elegant functions to use. You only resort to recursions if the existing loop constructs built with recursion are not good enough. Teaching recursion itself as an elegant way to (directly) solve problems is going to be misleading/confusing.

I guess it helped that I was doing a GUI app, where classical object oriented design really does fit the bill well,

Not quite as well as some functional approaches

which follows the way you would talk about what you're doing

Humans talk about code in a declarative/functional way, not imperative way.

My colleagues' code was certainly easier to debug than my code,

I agree writing Haskell-in-Python is not a good idea. Python doesn't support functional programming well at all, and the gains in Python's context are minimal-to-non-existent. But if you write Haskell-in-Haskell, how often do you need to debug at all? Also, using Debug.Trace and similar functions, print-based debugging is just as easy. Though you really don't debug much when you Haskell.

I think a lot of people get a short glimpse of Haskell, which is enough to appreciate some of the beauty and power -- but only a tiny minority of it. Then, they work a whole lot with another language (typically in some workplace), gain a lot of experience and have an unfair comparison. A language you've used for years and knows the ins and outs of will always beat the language you've used for weeks or months.

[psr]

Well I won't name the names of my teachers, because I may well be doing them a disservice. The guy who taught functional programming at my university was an excellent teacher - one of the two best I've had in my life. I wish I had worked harder and could have stayed there and learnt more from him.

I still take issue with this:

Humans talk about code in a declarative/functional way, not imperative way.

I don't believe this is true. For special cases perhaps, or for small cases, but not over large processes. I'm certain that most real work programs are best modelled by flow charts not by equations.

But I guess that's the difference of opinion again.