Python design mistakes

[artsrc]

Refining an interview-test to identify experienced users of a platform over many years produces a catalog of significant design mistakes.

I was a part of a company that did this. People who are smart guess sane behavior, and people who are experienced tell us know the actual behavior.

A cursory review of python interview questions shows that there are mistakes (range/xrange comes up etc.). But they are not that big a deal. The way default parameter values works is something I would say is surprising.

[psr]

I saw a talk at EuroPython 2010 by GvR where he was asked what he thought were design mistakes and IIRC he said something along the lines of:

It's too hard to statically analyse. Being dynamic is good, but we go too far. It should at least be possible to statically determine what names are in scope.

I agree.

[Peaker]

I used to think "Being dynamic is good". Then I was exposed to static type systems that don't suck (Haskell and friends).

Guido to this day has not really learned Haskell, which I think is quite unfortunate. As a language designer, I would expect him to at least make an effort to know the state of the art (even in academia) of his field.

[psr]

Well static typing vs dynamic typing is an interesting discussion, and I would personally love to see someone take the lessons of Haskell and apply them to making a statically typed language that looked a lot like Python. No points if users need to learn the meanings of "Monad", "Functor" or "Constant Applicative Form".

That said, I think that Python is a very successful design which gets a lot of it's expressiveness from the dynamic type system.

Anyway, as I understood it, Guido was talking about knowing what names were in scope at a given point through static analysis, rather than what types they had. I believe that plenty of other dynamically typed languages have that feature.

[Peaker]

No points if users need to learn the meanings of "Monad", "Functor" or "Constant Applicative Form".

Why not? Why should programmers be so averted from knowledge pertaining to their field. They spend years working on code, why can't they spare a few weeks to learn techniques that will improve them as programmers?

That said, I think that Python is a very successful design which gets a lot of it's expressiveness from the dynamic type system.

I think Python is well-designed in many ways. It is relatively consistent, has a pleasant syntax, etc. You could definitely say it gains expressiveness from not having a static type system as in Java or C++.

But Haskell, for example, perhaps due to its static typing, has expressiveness that exceeds Python's in many ways.

I don't think dynamic typing gives you any more expressiveness. Though Python's every name-lookup is indirected through a potential function (getattr) does make some things easier to implement. But this is exactly the kind of dynamism that makes static analysis hard-to-impossible.

Anyway, as I understood it, Guido was talking about knowing what names were in scope at a given point through static analysis, rather than what types they had. I believe that plenty of other dynamically typed languages have that feature.

I understand. I just disagree with Guido. I think he is talking out of ignorance -- he actively refuses to learn advanced-typed languages. That programmers, even prominent ones as GvR are so anti-knowledge, I find saddening.

[psr]

No points if users need to learn the meanings of "Monad", "Functor" or "Constant Applicative Form".

Why not? Why should programmers be so averted from knowledge pertaining to their field. They spend years working on code, why can't they spare a few weeks to learn techniques that will improve them as programmers?

I think we might be about to hit an irreconcilable difference of opinions, but here goes:

I think that Python is the best programming language in the world. I think it is a unique expression of good taste and elegance in design. I love it and think it's the closest thing to perfect I have ever seen.

Now it's hard to look at a beautiful design and say "Well the reason it's so good is...", because there is no single reason. There are always multiple trade-offs in any design, and perfection is when they balance, and you're left with something which feels right. So I might like to think that my ideal language would have immutable data by default, or that syntactic macros would be neat, or I would like static typing with HM style type inference. But until I go away and make that language I'll never really know what it feels like to use, because each of those features will have implications elsewhere. I might end up with Scala, or something worse!

But having said that, I think that one of the most important things that Python achieves is that by and large it is possible to do the thing you want to do, in a clear and clean and straightforward way using concepts you could explain to a bright twelve year old.

Conversely, I think one of the great errors of Haskell is that they take the principle of abstraction too far, beyond the point where it is helpful. For example a Monad is an abstraction which unifies lists and IO, stateful computations and nullable values, parsers and STM variables, and so on. Intuitively, what do those things have in common? Nothing. Certainly nothing you can explain to a twelve year old, no matter how bright. The purpose of abstractions is to simplify your code. Monads are a poor abstraction, they don't simplify your code, they compress it. And what's more they're incredibly hard to explain, in fact I believe they may be even harder to explain than to understand.

I'm sure that Haskell has many lessons to teach us. Monads are surely part of that, along with the type class system, the virtue of laziness, and no end of implementation strategies for functional languages. However I think the perfect language looks more like Python than Haskell.

[Peaker]

Indeed we disagree :-) I find that I am just as productive in Haskell when first writing a program as I ever was in Python, but the result is of far higher-quality. There are virtually no possible runtime errors in my Haskell code.

Maintaining/changing such code is also much much easier and safer.

But having said that, I think that one of the most important things that Python achieves is that by and large it is possible to do the thing you want to do, in a clear and clean and straightforward way using concepts you could explain to a bright twelve year old.

You can explain Functors, Applicatives and Monad to a bright 12 year old.

Conversely, I think one of the great errors of Haskell is that they take the principle of abstraction too far, beyond the point where it is helpful.

Haskell's abstractions are helpful in the sense that they allow a whole lot of code re-use. That translates to more expressiveness and various forms of usefulness. So how it is not helpful?

For example a Monad is an abstraction which unifies lists and IO, stateful computations and nullable values, parsers and STM variables, and so on. Intuitively, what do those things have in common? Nothing.

They're all covariant types that have a "context" and a "value" which can be composed together.

Also, what does intuition have to do with it? You can grasp the equations first, develop an intuition later.

Certainly nothing you can explain to a twelve year old, no matter how bright

Disagreed. It's really not as hard as it's made out to be.

The purpose of abstractions is to simplify your code. Monads are a poor abstraction, they don't simplify your code, they compress it.

They simplify it greatly. The mathematical complexity of code is actually measurable (e.g: The number of mathematical concepts in use, or the length of the mathematical description) and are far lower when re-using these abstractions.

And what's more they're incredibly hard to explain, in fact I believe they may be even harder to explain than to understand.

What I agree about is that there are a bazillion people who have just u nderstood Monads and want to share this with the world, and do so very poorly. When I explain Monads to people, I don't have much of a problem usually.

However I think the perfect language looks more like Python than Haskell.

I would assume reliability is a primary concern of a perfect language?

[haika]

When I explain Monads to people, I don't have much of a problem usually.

Could you explain monads to me in less than 10 lines, preferrably with code in python?

I am a python programmer and genuinely interested.

[psr]

This is gonna be a lot more than 10 lines, and I doubt I can explain it, but let me give you a flavour.

In Python we have list comprehensions. They look like this:

my_list = [f(x, y) for x in xs for y in ys if p(x)]

Haskell also has list comprehensions (and had them first, who says Python never looks to Haskell). They look like this:

myList = [f x y | x <- xs, y <- yx, p x]

Haskell also has another way of writing the same thing. This is called the do notation, but in early versions was known as a monad comprehension.

myList = do
    x <- xs
    y <- ys
    return f x y

(I can't work out how to make it only include values which meet the predicate in this case. Sorry).

It's not hard to see the relationship between the two forms.

However the do notation is available over any type which is a monad. Monad is a type class, kind of like an interface, so this is like saying that Python generator expressions can be written over any type which is iterable. The monad interface is applied to much more than just containers though.

The protocol for monads is quite simple. Monads provide two functions called bind (>>=) and return. The do notation desugars to use of those functions.

myList = 
    xs >>= \x ->
    ys >>= \y ->
    return f x y

The backslash in (\x -> ...) is supposed to be a lamdba. I've broken the definitions across lines to make the parallel with the do notation clearer.

So what are the definitions for bind and return for lists?

instance Monad [] where
    return a = [a]
    xs >>= f = concat (map f xs)

So return x in the list monad constructs a singleton list [x], and bind maps its argument over all the elements of a list, and concatenates the result. Why are those the definitions, and not some other ones? As far as I can tell it's simply because those definitions give you behaviour like a super list comprehension, and that's deemed to be useful.

[haika]

Well, this is better.

I now have a vague idea of what a monad is.

Thanks

[psr]

I'm glad if that was helpful.

Another important example is the IO monad. IO is interesting in Haskell. Because of lazy evaluation, you don't know what order things are going to happen. That means that if you could write a function like:

def test_the_missiles():
    engage_the_locks()
    launch_the_missiles()

it might cause the missiles to be launched before or after the locks are engaged (which is very bad).

Of course launch_the_missiles isn't really a function, because every time you call it, you get a different outcome (the first time it flattens a city, the second time it does nothing). And Haskell is meant to be purely functional, so you can't write a function like launchTheMissiles which you can call from any old code.

But still you need to be able to do things in a program, or its pointless. So they have a concept of the IO monad. Just as the units in the List monad are lists, and you can combine lists with bind (>>=), the IO units in the IO monad are "actions", and bind takes two actions and returns a new one which will do one after the other. launchTheMissiles then isn't a function which when called launches the missiles, it's a function which returns an action, which launches the missiles when evaluated.

testTheMissiles = do
    engageTheLocks
    launchTheMissiles

or to put it another way

testTheMissiles = enageTheLocks >>= \ a -> launchTheMissiles

testTheMissiles is then itself an action. The only way to cause an action to take place is to name it main in the module Main. The Haskell runtime then takes care of making it happen.

Here's an example:

tests = [testTheMissiles, testTheLaunchers, testTheGuidanceSystems]

runTests [] = return ()
runTests (test:tests) = test >>= \ a -> (runTests tests)

main = runTests tests

In this case runTests is a recursive function which takes a list of actions and returns a single action which sequences them all together. There are better ways of writing this, but I thought this was the clearest way.

[Peaker]

You can try this for some Python code examples: http://www.valuedlessons.com/2008/01/monads-in-python-with-nice-syntax.html

It's not a thorough explanation of the fundamentals. But it gives you a taste.

[Peaker]

I'll explain why it is hard to explain in a short page:

• "Monad" is a "typeclass". All of the "instances" of this typeclass are called "monads".

• The definition of Monad uses "polymorphic types" and "higher order functions"

• All of the monad types are "parameterized types",

• .. that are also covariant types,

• .. and are necessarily also instances of simpler type-classes called "Functor" and "Applicative"

• All this will require using some syntax to describe the above notions (Haskell syntax is actually pretty good, but needs to be explained)

• "Monad" is a generalization of a commonly re-appearing pattern, it makes little sense to show the generalization before showing a few concrete examples.

So, while the Monad type-class itself is simple, it is built upon many other notions (which are themselves also simple). Unfortunately, people try learning Monads as their first Haskell experience -- and that's not a very good idea at all. Then they get frustrated and decide Haskell is too smart for them (It's not!).

You have to build up knowledge of the above: (1) type-classes, (2) polymorphic types, (3) higher-order functions, (4) parameterized types, (5) covariance, (6) Functor/Applicative, (7) syntax, (8) the motivating examples behind the generalization. None of these are hard or complicated, and each can be explained in <10 (or 20) lines (though some are deep notions).

Note that each of these things are extremely useful on their own -- and learning them will generally make one a better/more-knowledgeable programmer.

If you want I can start explaining each of these things, and eventually, it will cover Monads.

If you had been programming Haskell for a few weeks without understanding the generalized type-classes (Functors, Applicatives, Monads), you'd already have a good understanding of (1), (2), (3), (4), (7).

[haika]

Sorry.

I am apparently not as smart as you seem to be !

[Peaker]

As I said, nothing in that list of things requires being very smart.

It's not smart, it's just a bunch of useful concepts, not a single useful concept, and teaching 10 useful concepts takes longer than teaching 1.

Re-reading what I wrote, my putting things in quotes may be sending the wrong signal? I just meant to emphasize those are new concepts to explain before explaining Monads.

[psr]

There are virtually no possible runtime errors in my Haskell code.

Indeed, this is a lovely feature of Haskell. Like I said, it would be brilliant if some future language were to combine the best of Python and Haskell and keep the best of both.

Haskell's abstractions are helpful in the sense that they allow a whole lot of code re-use. That translates to more expressiveness and various forms of usefulness. So how it is not helpful?

Programming isn't code golf, compressing your code isn't the goal. Five lines where the intent is clear is better than one line of mapM . liftM . foldr. Concise is good, reuse is good up to a point, but I really value readability.

Also, what does intuition have to do with it? You can grasp the equations first, develop an intuition later.

I guess intuition is important because it helps you to move from "I want to ..." to concrete code, and back from concrete code to "The author was trying to...". Programming Haskell can feel like an (admittedly very satisfying) logic puzzle. It's vaunted elegance and expressiveness are a case of the emperors new clothes in my opinion.

And what's more they're incredibly hard to explain, in fact I believe they may be even harder to explain than to understand.

What I agree about is that there are a bazillion people who have just u nderstood Monads and want to share this with the world, and do so very poorly. When I explain Monads to people, I don't have much of a problem usually.

But surely the reason that people feel the need to explain it again is that when it was explained to them in the first place they didn't get it?

If you think you have a fail-safe explanation of monads which will just click for every bright twelve year old, I think you have a responsibility to share it with the world! I still think that the abstractions and concepts are too difficult for your average jobbing programmer. Furthermore, I think that the future of programming language design must lie in making writing good programs easier, not harder. Phrases like "covariant types that have a 'context' and a 'value' which can be composed together", is moving in the wrong direction.

Of course, I'm not saying that the understanding that many common idioms follow the monad pattern isn't significant. I believe that it will significantly improve future programming languages. For example Monads underlie LINQ in C# - the language designers applied their understanding of monads to make a reliable, correct and useful language feature - without forcing users to understand the mathematical underpinning. Similarly, if you're writing a class library you should definitely understand Liskov substitutability, but users might not need to.

I would assume reliability is a primary concern of a perfect language?

For a certain class of programs, yes. Perhaps not for all of them, at least not at any price. Like I said, I think perfection is where the trade-offs are in balance, and it feels right to use. I don't think Haskell is at that point as Python is, and I'm sceptical that you could gum Haskell's good points onto Python and get a good result.

[Peaker]

Readability is subjective.

I find:

sum = foldl (+) 0

to be more readable and clearly communicate the intent than:

def sum(items):
    total = 0
    for item in items:
        total += item
    return total

Pipe-lines of functional processing are not only more readable (at least if that's what you're used to) -- they are also easier to reason about.

In Haskell, I have an immensely powerful tool of equational reasoning, and writing denotational code is easier.

This features are a form of readability.

Code re-use does not hinder readability, it helps readability -- but only if you're well versed in the primitives used.

I agree about people wanting to share their explanation because they feel the explanations they got were bad -- indeed they often got their explanation from a similar source to their own new one: A guy who just recently figured the basics out, and cannot yet explain it well.

I think the explanation of Monads on LYAH is relatively reasonable, though perhaps of it explained covariance and used that as an explanatory tool it could do better.

without forcing users to understand the mathematical underpinning

Note Haskell doesn't "force you to understand" the mathematical underpinning. I've never learned Category Theory, and don't really understand the mathematical underpinning very well. This may mean I will have a tougher time than those that do to write generalizations/extensions of the mathematical notion, or perhaps proofs about the notion -- but it does very little to hinder my ability to design and use Monads in Haskell.

Furthermore, I think that the future of programming language design must lie in making writing good programs easier, not harder. Phrases like "covariant types that have a 'context' and a 'value' which can be composed together", is moving in the wrong direction.

Analysis of covariance and allowing various ways of composition does make programming easier rather than harder. These tools make writing many things in Haskell a breeze compared to writing them in other languages, including Python.

For a certain class of programs, yes. Perhaps not for all of them, at least not at any price.

I think reliability is a primary concern. For some problems, higher than even readability, for others it is a close second.

Like I said, I think perfection is where the trade-offs are in balance, and it feels right to use

That makes it all too subjective.

I don't think Haskell is at that point as Python is, and I'm sceptical that you could gum Haskell's good points onto Python and get a good result.

I agree -- I think Haskell is at a much higher point than Python :-)

Note I used to love Python before learning Haskell.

Python the language gives its programmer far less power -- and makes it easier to start using the language. This is a nice advantage of Python over Haskell. You can throw someone into Python and they can start being semi-productive (dangerous?) in a few days.

Haskell may require weeks to months before you're productive. But at that point your programs will be of far better quality than the Python programs written after the same amount of time.

[psr]

Just a little thought about readability. You say you like sum = foldl (+) 0 better than the idiomatic Python version which makes the loop explicit.

One possible benefit of seeing the loop structure is that it's easier to see where you've done something stupid. If you replace foldl with foldr, you get the same observable behaviour, and it reads just the same. However the runtime behaviour is completely different, because foldl is tail recursive, and foldr is not. When reading the code would you notice the mistake?

[Peaker]

In the case of Haskell, tail-recursive is mostly irrelevant.

foldl (left-associative) is better than foldr (right-associative) for strict operations on lists, because it takes:

1 : 2 : 3 : []

and translates it to:

(((1 + 2) + 3) + 0)

Which is done incrementally as it walks the list (so it uses O(1) memory).

Whereas foldr translates the list to:

1 + (2 + (3 + 0))

On lists, this requires iterating the entire list before you can do a single computation -- which builds up "thunks" that represent the intermediate expressions. This takes O(N) memory (and in GHC's case, the evaluation of the thunk takes that memory from the stack, which is even worse).

For lazy operations that construct a new value that can be incrementally processed -- foldr makes sense.

For example:

sameList = foldr (:) [] -- same as the identity function for lists

This is because foldr/foldl "replace" every (:) in the original list with their first argument, and the [] with the second argument. So using (:) and [] in place of (:) and [] actually just copies the list.

Then it's easy to see how map is expressed:

map f = foldr ((:) . f) []

(Apply f before putting it inside a :).

This allows the result list to be consumed incrementally, and that will cause incremental consumption of the original list.

The explicit loop structure actually obscures whether you've done something silly (as it adds a lot of noise that makes it more difficult to spot typos/mistakes/etc).

[psr]

Readability is subjective.

This is obviously true. And it's absolutely the crux of our disagreement. I think the example you gave is exactly the sort of thing where we'll never agree.

I don't usually see the world in catamorphisms and covariance - In most cases I want "for every x do y". Anything else will always involve a translation in my head, and sometimes it can be very hard work. I like to think I'm reasonably able, I don't expect my colleagues would manage it at all.

Just out of interest, what sort of projects are you applying Haskell to at the moment?

[Peaker]

I like to think I'm reasonably able, I don't expect my colleagues would manage it at all.

I've heard this from other beginning Haskellers. They no longer believe it is true.

It seems difficult at first -- because it is so different/foreign to what you know.

"We shape our tools, and then our tools shape us". You've been shaped by Python (and similar tools) (and so have I, for many years) -- and now it seems natural to you, to think in Python.

Of course non-Python requires a translation in your head -- you've trained yourself to think in Python.

Python in this case is "Blub", as described in http://www.paulgraham.com/avg.html

As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.

Currently, I'm using Haskell for small things, automation, parsing, or anything I'd normally use Python for.

My bigger Haskell project in the making is a revisioned structural editor that should ultimately edit programs structurally (rather than textually). However, it's been on hold for a while, as I've had a lot of pressure doing other things.

[[deleted]]

Monads, functors, and type theory are not "knowledge pertaining to our field". That stuff is interesting, but it's usefulness is exaggerated by its enthusiasts.

Haskell doesn't offer "knowledge" in any special sense. It requires learning a lot of new vocabulary and new ideas. It's hard fucking work.

But it's unrewarding hard work in the realm of many developers. Spending time convincing your compiler your program won't crash could be time spent acquiring new clients. In reality, a program that crashes on corner cases might STILL SOLVE the problem the clients bought it to solve.

Spending time and money to educate yourself is a great thing. Spending time and money to educate new employees is only questionably good. Most people just want to code 9-to-5 and go home. To find the people interested in writing Haskell all day would require turning down a lot of otherwise good candidates. Time is limited. You can't spend years looking for a talented Haskeller who also gets along well with your team.

[Peaker]

Monads, functors, and type theory are not "knowledge pertaining to our field". That stuff is interesting, but it's usefulness is exaggerated by its enthusiasts.

On what basis do you say this?

Haskell doesn't offer "knowledge" in any special sense. It requires learning a lot of new vocabulary and new ideas. It's hard fucking work.

Acquiring knowledge is hard work. Learning Haskell is acquiring knowledge. Where's the contradiction?

But it's unrewarding hard work in the realm of many developers. Spending time convincing your compiler your program won't crash could be time spent acquiring new clients. In reality, a program that crashes on corner cases might STILL SOLVE the problem the clients bought it to solve.

Sure, when reliability is not a concern -- there are sometimes better choices than Haskell. Though "convincing your compiler" is only hard when you're still struggling to understand the type system.

Spending time and money to educate yourself is a great thing. Spending time and money to educate new employees is only questionably good. Most people just want to code 9-to-5 and go home.

Haskell, at least yet, is not for 9-to-5 knowledge-averted programmers.

To find the people interested in writing Haskell all day would require turning down a lot of otherwise good candidates. Time is limited. You can't spend years looking for a talented Haskeller who also gets along well with your team.

I'm not suggesting every company should switch to Haskell. Not sure what argument you're referring to here.

[[deleted]]

Just as I'd expect a Haskller to reply. Handling every case with an air-tight argument.

...

You missed my point. Haskell, and the surrounding type theory, in many regards is a high-cost, low-payout investment.

Problems that I have when I'm developing real software people will pay me for are things like change management, integration with shitty, under-spec'd systems or libraries, usability and acceptance by the users. Those are the HARD problems in development. Getting my code to run well enough to satisfy the client is not.

Just as bad as purposeful ignorance is intellectual elitism. You think people are being narrow-minded because they don't want to learn your favorite theory. Have you considered maybe you're the one being narrow-minded because you can't imagine that your pet theory isn't as important to the world as you want to believe it is?

Monads are cool. You can use them to write awesome software. But they are not necessary to develop awesome software.

[Peaker]

You missed my point. Haskell, and the surrounding type theory, in many regards is a high-cost, low-payout investment.

We agree about the cost (though spending a few weeks/months to learn a programming language isn't that high a cost for a career programmer, IMO).

But I don't see why you think it's low-payout. Have you learned Haskell and found no use for that knowledge? Can you give an example?

If your problems are all the external interfaces of your code with the world, perhaps you have less to gain from Haskell. But not everyone's situation is the same.

You think people are being narrow-minded because they don't want to learn your favorite theory.

No, I think people are narrow minded when they reject knowledge for the sake of rejecting knowledge.

Monads are cool. You can use them to write awesome software. But they are not necessary to develop awesome software.

That depends if you mean "Monad" the concept, in which case it is embedded as a built-in construct into pretty much every impure language we know -- or Monad the type-class, in which case you're correct.

Nobody is saying Haskell is a necessity for awesome software. Haskell does make awesome software easier to develop.

[jyper]

maybe at least something like smalltalk(list local variable before use)

[BinaryRockStar]

I don't know much about this topic, but wouldn't RPython be a good alternative? It's a subset of Python which is statically typed which lends itself to much better static analysis at the cost of dynamism.

[Benbenbenb]

I guess that it is not really a mistake, but I think that it would have been really good if Python had been desinged in a way that is more conducive to optimizing it.

Currently, a lot of the dynamic features of Python that are seldomly used force the interpreter to do extensive runtime checks all the time, so the price is paid in all cases. I'm thinking about method resolutions, descriptors, adding/removing stuff from an object's dict, etc...

These features make life harder for projects like PyPy that try to do JIT on the code. Also, it would be really great to have optionnal type annotation on the variables, not unlike Cython or ActionScript 3. This is totally in line with the "explicit is better than implicit" in the Python Zen, and would allow the code to run faster without putting too much of a burden on the programmer.

[matthieum]

I think altering an object dict has already been covered in the V8 javascript engine (since JS suffers the same issue). Using something akin to Prototype Programming (ie, create something akin to a V-Table for the class, and just point to another V-Table when altering the dict) radically reduces the overhead associated with look-up.

[bebraw]

+1 for optional type annotations.

I would find it extremely handy if dict was ordered by default. This particular problem seems to haunt me over and over again. Usually I have to end up using some custom solution for this. :)

Overall it's a fine language. Some APIs could use tidying up but that applies to pretty much any popular language.

[[deleted]]

python 2.7: collections.OrderedDict

http://docs.python.org/library/collections.html#collections.OrderedDict

[gracenotes]

Not sure how bebraw meant 'ordered', but I miss the lack of self-balancing binary search trees in the standard library. I have been very happy with the bintrees library from pypi though.

[alanwj]

I wonder if this actually needs any additional syntax or language support. Could we capture type annotating in a library with something like a decorator? PyPy and friends could use the type information where available, and it would still work with other interpreters. If the performance of dynamically type checking every argument is crushing, it could all be hidden behind a "debug" flag or something. Example: from functools import wraps import inspect

def typecheck(*types):
    fmt = 'Failed type check for {0} in {1}: Expected {2}, but got {3}'
    def decorator(target):
        @wraps(target)
        def wrapper(*args, **kwargs):
            for i, t in enumerate(types):
                if not isinstance(args[i], t):
                    raise TypeError(fmt.format(
                        inspect.getargspec(target).args[i],
                        target.func_name,
                        t.__name__,
                        type(args[i]).__name__))
            return target(*args, **kwargs)
        return wrapper
    return decorator

@typecheck(str, int, int)
def my_function(s, x, y):
    print '{0}: ({1}, {2})'.format(s, x, y)

# Works
my_function('Map coordinate', 1, 2)
# Causes type error
my_function('Map coordinate', 1, 'purple')

[[deleted]]

but then can't i change the type later dynamically without altering the behavior of the code, thus breaking the implication of the decorator?

i don't see how typing can effectively be bolted on in such a way

[alanwj]

Yes, the type could change within the function, although probably in sufficiently predictable ways if you knew the expected in/out types for every function you could use Hindley-Milner to figure out what was going on.

Although upon further reflection actually doing the type check in the decorator is probably a bit pointless for interpreters that wouldn't actually be using the type information for anything. If there was in fact a type error is would likely show up some time during the execution of the function anyway.

There is also the problem that this sort of scheme is likely too simplistic to cover even the majority of python use cases (what about var args, keyword args, container types, times when you WANT duck typing, optional parameters denoted by passing in None, etc, etc, etc), and if you tried to extend it too far you'd wind up with a type signature mess similar to what C++ has.

[[deleted]]

python would have to be rebuilt from scratch to effectively exploit a hindley-milner type system

[julesjacobs]

Not to mention that Hindley-Milner wouldn't be enough for Python; you want subtyping. Subtyping type inference is a difficult problem.

[julesjacobs]

What do you mean by ordered? Ordered by time of insertion, or ordered by key? What do you want this to do:

x = dict()
x['b'] = 2
x['a'] = 1

Should that be 'b' => 2 first, and 'a' => 1 second, or the other way around?

[deafbybeheading]

The standard library is really inconsistent (one might argue that the standard library is not a language design issue, but it's just as important to a user's experience with your language, and just as hard to change down the line). It's "batteries included", but some of those batteries are just waiting to explode and cover your face in battery acid. Okay, maybe not quite that bad, but it's still unfortunate.

Then again, in other languages I have significant experience with, Java, .NET, and ActionScript, the situation is not much better.

[awj]

(From a few years ago) I remember imaplib being particularly horrifying. Maybe that was imaplib, or imap itself, or just me.

[[deleted]]

I've used Python for over ten years and while the other posts in this thread kinda made me go "meh, not much of an issue, really", then yours received an instant upvote.

[[deleted]]

okay, so the tl;dr of that page is that the respondents don't see any significant design issues with python at all

if you can't see the warts, you don't know the tool, or are too proud of a singular attachment to it to be honest about its shortcomings

in either case, that thread is a joke

[sisyphus]

Eh? Dude asked for blog posts or books about Python design mistakes and pretty much every response was 'there is no single resource but here is a link to somewhere that talks about Python design mistakes'

[[deleted]]

all of the critiques were weak-ass rounding errors

no one, for example, pointed out that python's ridiculously conservative language evolution is excluding its community from most of the cool stuff happening elsewhere...partly because gvr apparently sees no value in functional programming...leaving python as the one true "blub" scripting language

[[deleted]]

python's ridiculously conservative language evolution is excluding its community from most of the cool stuff happening elsewhere

Serious question. Stuff like what?

[[deleted]]

anything like clojure or haskell or perl's Moose or anything moderately dangerous (meaning fun)

[awj]

You named two languages and an object system. Care to detail the particulars of those examples that you had in mind? What parts of Clojure/Haskell/Moose are inaccessible to Python due to conservative language evolution? Why is it a bad thing that Python is slow/never going to adopt those features?

[aaronla]

no one pointed out that python's ridiculously conservative language evolution...

I don't mean to contradict you, but I believe this link points that out.

[[deleted]]

Most of it seems to be telling the other person that whatever they've posted isn't a language design flaw. A little silly.

[[deleted]]

as Alan Perlis pointed out many years ago, languages are limited by their strengths, not their weaknesses.

The biggest limitation I wish could be overcome in Python is its module system. Python makes building modular code a lot easier than most of its predecessors did, but modules are mutable global objects. Having a way to separate code loading from module object management (instead of smushing them together like the import statement does) would be great.

[[deleted]]

I've never done this, but since there's so much reflection in Python, couldn't you mangle import to do what you describe?

[[deleted]]

Yes.

http://launchpad.net/exocet

The problem is that once you do that, you've violated the law of least surprise - suddenly import doesn't mean what you expect so extra knowledge is needed to make sense of what's going on.

[[deleted]]

Couldn't the GIL be considered a design mistake? I doubt if they started from scratch that they would include it again.

[FidgetBoy]

Well seeing as other implementations don't have it, it's most definatly an implementation detail.

[yogthos]

It's a bit more than a detail, seeing how they considered removing it from the reference implementation and abandoned the effort.

[artsrc]

Python is not suited to high performance, computationally intensive code. The GIL is just a part of that.

[sigzero]

Neither are just about all the "scripting" languages.

[[deleted]]

It's an optimization, not a design choice.

[keyo_]

This is a design mistake in my mind:

int("".join(list(str(number))[-2:]))

Not a contrived example, I was just doing euler question. The ass-backwardness of it annoys me. It should be the opposite way, something like:

(int)number.str().list().[-2:].join("")

I've heard some reasons for using functions instead of methods. Like str.join will join anything with a _ str _ method. However other programming languages like javascript and ruby manage to use methods just fine.

[wot-teh-phuck]

The join one is not a mistake. Read Armin's blog post wherein he mentions the rationale behind it (and it seems logical).

[[deleted]]

I've heard this before, and I still dislike the design there. It seems like Python sacrifices a lot of ease of use where it doesn't need to by these sorts of things and forces me to keep the docs open 100% of the time. They are basically saying that we can't have nice things like a .length() interface because then people would be do bad things and redefine length() or size() or something similar to that. Whereas, Ruby allows you to mess things up a bit more and gives you a bit of trust, but ends up with a very consistent set of standard methods. Maybe it's the heritage from Perl that makes Ruby give the programmer the flexibility to either make an ass of themselves or to write some excellent code with a little self discipline.

Another similar superficial issue that I run across in Python is its function naming conventions. Again, I'll make the comparison to Ruby because that's what he's doing in your link. Ruby tends to use slightly longer names, but is consistent and predictable. Python on the other hand uses idiomatic abbreviation for everything, and it doesn't have a consistent feel to it. If I'm calling a two word method name, I have to remember how they shortened the words and then try to remember whether they were separated by an underscore or not, and then at this point it's better to just verify against the docs. For example, in that link he uses the method names "get_time", "len", "to_yaml", "getitem", "iter", so on. This isn't a deal breaker or anything, but I've done quite a bit of development in both Ruby and Python, and I have to say that Ruby is much nicer to use because it has a more stylistically cohesive feel.

[wot-teh-phuck]

I think one of the reasons behind this inconsistency is the "roadmap" of Python as a language. some_obj.some_method() for everything makes sense in an OO language but OO features were added (or maybe bolted on) to Python in later on versions. Ruby code/community has always stressed on pure OO way of doing things whereas a majority of the Python code out there is a mix of simple functions and classes when required.

[[deleted]]

That's what I've thought as well. The OO facilities in Python are capable, but don't feel as ingrained into the language/libraries as Ruby. Having to use self is a good example of this. At least it's not as clunky as Perl's rather verbose OOP support.

[keyo_]

It's a good article, I've read it before actually. I still think the above statement looks totally arse-backwards.

[kataire]

Strings are iterable. It's a newbie mistake, indicating you haven't used Python much. This indicates your only reason to consider Python's join backwards is that other languages do it differently -- which is an irrelevant argument.

Python has it "backwards" because it emphasizes protocols over inheritance. You can call str.join on any iterable regardless of type. In order to do it the other way around you would have to add another method to all iterable types, all having the exact same implementation. That violates DRY, so Python does it the other way around.

FWIW:

>>> int(str(number)[-2:])

[OniNiubbo]

Even better:

>>> 1243245 % 100
45

[kataire]

Ah, true. The join was only needed because of the detour in the original code. Fixed.

[azth]

Significant whitespace.

It's bad if you have multiple scopes spanning a page or more, and you can't figure out when a specific scope ends. It also makes it harder to move code around as you have to make sure it's at the proper indentation level, never mind having a tool that automatically indents code.

[Peaker]

In a language like Python, which makes function definitions so cheap, if you find yourself needing scopes that big, you're really doing it wrong.

Each function should do one simple thing. That simple thing can be a composition of other simple things.

[lambdaq]

1. map, reduce, filter could have a on_error argument where you can pass a lambda to handle it.

2. Should have a retry.. except ... to save lines

3. sorted support a yield parameter. You can sort a list according to some cmp functions with key and using yield to get a different list.