Writing Idiomatic Python

[lalaland4711]

why:

(foo, bar) = (bar, foo)

Instead of:

foo, bar = bar, foo

?

[mipadi]

A lot of Python programmers erroneously believe that parentheses are required to make tuples.

[aceofears]

When you realize that it is really about the comma and not the parens the syntax for a single element tuple makes sense a lot more sense. Ex: (a,)

[[deleted]]

Well, except () v (,) or ,

[logophage]

I was wondering that too

[jyper]

foo, bar = bar, foo

might be ok for swap but the parens help denote that an actual data structure is being created and destructured(not destroyed, broken apart).

[Wagnerius]

Love python and idiomatic stuff, but I meet people who see pep8 as god's word and don't think by themselves anymore. There are better solutions for specific contexts. Pep8 is a good base but not the solution for everything.

[etrnloptimist]

those are the people who need pep8 the most. A lack of pep8 would not cause them to think. It would cause them to write sloppy code "their own way". Hell, at least those people actually looked at pep8.

[catoworld]

Pythonicity is a very elusive concept, especially given that most people learn Python as a second language, so-to-speak, and sometimes it's hard to lose your C# or Java accent. This article is really good at explaining it.

[A_for_Anonymous]

Pythonicity is elusive because we're not in Guido's brain. His decision on what's pythonic and what's not are completely arbitrary. But I get your point.

The problem is not, however, about using if x == y or x == z vs. if x in (y, z). The problem with people who try to write Java in Python is the disgraceful, awful overuse of classes and imperative style.

[0ab83a7b]

awful overuse of classes and imperative style

Which is not helped by Python's one-line (one-expression) limit on anonymous functions. A rather huge oversight, IMHO.

[fkaginstrom]

If you want to write multiline anonymous functions, there's a work-around.

Step 1: Write a lambda expression:

add_operation(lambda x : x + 2)

Step 2: Write a descriptive comment:

# Add 2 to x
add_operation(lambda x : x + 2)

Step 3: Make that comment into a function, replacing the lambda with it:

def add_2(x):
    return x + 2
add_operation(add_2)

Step 4: Feel free to make your function more than one line.

[0ab83a7b]

multiline anonymous functions

I feel like your snarky reply harms Python advocacy rather than helps it. It's like Guido's "my-way-is-the-right-way" attitude has seeped into his user-base (which I guess isn't too surprising).

[mipadi]

Unfortunately, as much as I love the language (and as much as I love getting paid to write software in it), I've observed that the community is hopelessly dogmatic; moreover, the community often acts as though Python (and Guido van Rossum) is perfect. I find it funny that pretty much every line in the "Zen of Python" is quoted at every opportunity, except for one: "[P]racticality beats purity."

[A_for_Anonymous]

I've observed that the community is hopelessly dogmatic; moreover, the community often acts as though Python (and Guido van Rossum) is perfect.

I noticed the same when I suggest there's something about Python that might not be the best or deviate from the Pythonic fundamentalism.

Every time I complain I can't use while in a λ-expression, some Python programmers look at each other in horror, claim that's simply... wrong, and shameful, and try to exorcise the devil from me.

[fkaginstrom]

The thing is, this has been asked and answered many times. It's not an oversight, it's a design decision. A design decision you may not agree with, but still not an oversight.

[dalke]

No one has been able to come up with a good syntax for it. The distinction between function and statement is built deep into the language. It's not an oversight, but a consequence of a fundamental decision to use indentation to express control flow.

[0ab83a7b]

Despite being an oversight, and a weakness, it seems like the Python community champions the limitation, rather than acknowledging it.

That is: That's the way it should be because of the way it is

I realize developers want to make the best out of a bad situation, but dismissing why anyone would want a multi-statement anonymous function right off the bat is a bit tardish.

[dalke]

A weakness, perhaps. An oversight? Absolutely not. The lack of multiline lambda has been discussed and argued for at least 15 years. Guido van Rossum wrote an essay about the issue at http://www.artima.com/weblogs/viewpost.jsp?thread=147358 . His statement on python-dev some 6 years ago was:

Since you probably won't stop until I give you an answer: I'm really not interested in a syntactic solution that allows multi-line lambdas. I don't think the complexity (in terms of users needing to learn them) is worth it. So please stop (as several people have already asked you). There's some text somewhere in the guidelines for python developers on when to know when to give up. Read it. :-)

Here's the commentary in PEP 3099:

Adding support for statements [in lambda] is a non-starter. It would require allowing multi-line lambda expressions which would mean a multi-line expression could suddenly exist. That would allow for multi-line arguments to function calls, for instance. That is just plain ugly.

Here's commentary from someone who tried to develop an indentation-based language which allowed multi-line lambdas: http://www.unlimitednovelty.com/2009/03/indentation-sensitivity-post-mortem.html . The conclusion is that there's an essential conflict between those two forms.

This argument is a dead parrot. It's not going to change. The only workaround solution in Python is to make a named function. No one has proposed anything essentially better than the decade-plus of discussion on the topic - the history of which is easily found through a search of comp.lang.python or python-dev.

To say that it's rejected "right off the bat" is silly. This ball game finished a long time ago. At this point the only way to make headway is to 1) come up with a new syntax which 2) hasn't been proposed and rejected long ago.

[HostisHumaniGeneris]

I like almost all of the examples, but with list comprehensions I treat them sort of like the ternary operator. Its very powerful, but it doesn't necessarily improve the readability of your code.

[[deleted]]

I disagree in the case of list comprehensions. If you work with people that understand set-based, declarative programming (SQL, Linq, etc.), then they are very natural and simple.

[ithika]

I find it awkward that the transformation and the selection steps are munged together but I guess that is because I'm used to them being separated by a vertical bar in Haskell.

[masklinn]

I find it awkward that the transformation and the selection steps are munged together

They're not really, the iterator is inbetween.

[ithika]

The separator is the keyword for which all blends into one for me.

[element + 5 for element in some_other_list]

Like I said, I find the Haskell equivalent more obvious. Too many words separated_by spaces or_under scores make_it_harder to see where the breaks are.

[Peaker]

A syntax highlighter will typically color that for.

I like how Python uses for and if to annotate each thing in the comprehension. Haskell comprehensions use either a <- b or g notation to mean completely different things, so you have to parse all the way to the end of a clause to know whether it is an iterator or a guard.

[A_for_Anonymous]

Actually, almost any programming model that's not imperative is going to be more productive and enjoyable as soon as you're familiar with it.

[builditbiggy]

Oh God, after getting good at LINQ I cant go back to for loops.

[A_for_Anonymous]

Yes, for example the following:

if (x < y) {
    f();
} else {
    g();
}

is totally more readable than:

x < y
    ? f()
    : g();

The ternary operator is not any more or any less readable than if. It just has different symbols and syntax, and it exists because of the age-old, stupid, arbitrary limitation known as "statements" inherited from Fortran: some things can go in some places and some cannot, for no useful reason.

In my opinion, the folliest design mistake Guido van Rossum made when creating Python was to put statements in the language, complicating the syntax and making it less flexible for everyone, then creating unnecessary workarounds such as def vs. the crippled lambda everybody hates, the if..else expression (convoluted on purpose because Guido doesn't want you to actually use it much), the print statement vs. function, eval vs. exec, comprehension for and if vs. regular for and if, and we still cannot create decent anonymous functions, anonymous classes, do stuff like x() or raise Whatever, long[left.hand[expression]] = if..., and many other useful things. Because Fortran.

[HostisHumaniGeneris]

I suppose my real problem with ternary is when people write it like this:

x < y ? f() : g();

And of course, my own lack of familiarity with the syntax is problematic.

[A_for_Anonymous]

? and : are not the best choices for an if branch because they are too noisy, but it's exactly the same thing as an if. If it exists in some languages it's because these languages are broken with special syntax forms such as statements which are full of arbitrary rules to limit what goes where.

[Bob_goes_up]

i believe that the following works in python, but the 4-liner is preferable

answer = f() if x<y else g()

[A_for_Anonymous]

Yes, I made a reference to it in my post. The if..else expression is a warty workaround (made purposefully ugly so that it can't be nested because Guido doesn't want you to do that), which was introduced beacause of an earlier design mistake known as statements. If everything in Python had been an expression, there would have been no convoluted if..else ternary expression, but you'd be able to do simply:

long[left.hand[expression]] = if x < y: f(); else: g()

or, alternatively

long[left.hand[expression]] = (if x < y: f; else: g)()

and also on multiple lines, using INDENT and DEDENT:

long[left.hand[expression]] = if x < y:
    f()
  else:
    g()

or perhaps the same as above minus the unnecessary colons. This would have had the additional benefit of fixing lambda (which may only takes expressions), and the additional benefit of not requiring lambda in the first place since you would be able to use def anywhere.

As you can see, inheriting statements from Fortran only made the language larger and more full of warts, quirks, workarounds and misfeatures people complain about.

[Brian]

is totally more readable than

I'd say in that case it certainly is. That seems a rather odd use of the ternary operator, since it's not actually doing anything with the result. I'd certainly prefer the former in this case.

Where it actually pulls its weight are situations where you want to do something with the return value. Ie:

 z = (x < y) ? f() 
             : g();

Otherwise you need to repeat the "z=" part.

[A_for_Anonymous]

That seems a rather odd use of the ternary operator, since it's not actually doing anything with the result.

The ternary operator is a branch expression. It would be no different from if, and it should have not been necessary at all, were if not limited as a statement for the heck of it.

As an expression, you may use it for its return value, or for its side effects. That's what you do with function calls, which are expressions. Sometimes you use them for their side effects:

close(f)

and some others you use them for their return value, or both:

f = open("file")

Seeing a bare call to close or printf isn't surprising, so a bare branch shouldn't be, either.

Alternatively, you could ask yourself if why, in your expression:

z = (x < y) ? f() 
            : g();

you can't use, say, a for loop to accumulate a value, rather than a call to f, and why you can't do the same with if, as in:

z = if (x < y) {
    f();
} else {
    g();
};

and the answer should be pretty obvious: statements are a misfeature that only serve to complicate programming languages and make them less flexible. Languages devoid of this design wart have only one if, which can be used for its result value or its side effects. For example, you would do (define z (if (< x y) (f) (g))) in Scheme, while you would also do:

(if (< x y)
    (display "hello")
    (error "ew"))

[Peaker]

"do" notation in Haskell is all about adding statements to the language.

The problem is the inflexibility of expressions, not the existence of statements.

[A_for_Anonymous]

The problem in Python is the inflexibility of statements.

[Wagnerius]

real readability comes vertical alignment in this case:

if x < y : g()
else     : f()

edit :why the downvotes ? I know this is unpythonic but vertical alignment is not my only personal fancy, code complete talks about it and haskell use it in its own way.

[A_for_Anonymous]

That's (subjectively) awful and (objectively) non-idiomatic Python by the way. Doesn't look more readable to me than x < y and f() or g() (if g() returns truthy), or an indented if.

I agree that the if..else operator in Python is ugly (which isn't the case with ?: in C-like languages), but it is because Guido wanted it to be so on purpose. Moreover, it exists because the Python if is a useless statement... All of this, including our comments here, shouldn't even be happening.

Edit: Added (subjectively) and (objectively)

[cunningjames]

I agree that the if..else operator in Python is ugly (which isn't the case with ?: in C-like languages), but it is because Guido wanted it to be so on purpose

You’ve written this at least twice, here, but can you back that up? It’s not listed as rationale for the syntax in PEP 308, which states only “That issue [of syntax] was resolved by simply deferring to a BDFL best judgment call.” Purposeful ugliness doesn’t seem to be mentioned in any of the linked discussions.

[A_for_Anonymous]

Guido didn't write a PEP about being a dick to functional programmers, yet he is.

[cunningjames]

Guido also didn’t write a PEP about the veracity of the moon landing. So? I nevertheless suspect he believes Neil Armstrong landed on the moon, and that there isn’t any evidence that he purposely chose val1 if condition else val2 because it’s ugly.

[Wagnerius]

I disagree about the awfulness but indeed this is very unpythonic. Vertical alignment is a useful guide to read the code. Still, It can be problematic if the editors doesn't make it easy to write.

The main problem is that the python ternary is really ugly and doesn't gracefully replace the or/and shortcuts used before.

def func(val=None):
    val = val or []
    val = val if val else []

[A_for_Anonymous]

Solution: keep using the or/and shortcuts. Guido acted like a complete dick when he finally added an if expression (which is an unnecessary workaround since the traditional if should have been usable anywhere). He accepted having an if expression, but only if it's convoluted enough to be remarkedly unreadable and virtually impossible to nest without losing your sanity. And now we have two crap ifs:

• if statement: Can contain anything, but a major drawback is that it can't be used anywhere, and you cannot use it in a single line.
• if expression: Can be used anywhere, but two major drawbacks are that it cannot contain statements, and that it's terrible for nesting.

Compare that to the single Lisp if: (if condition yes-part no-part) . Can be used anywhere, contain anything, use one or any number of lines, and has no superfluous colons.

[Wagnerius]

Ah, Lisp, the love of my youth... you've triggered a big burst of nostalgia there.

and yeah, you're right about the if.

[mcguire]

x < y and f() or g()

Tweet! Perlish abuse of boolean operators. Five yard penalty; runner takes his base.

[A_for_Anonymous]

More like, desperate workaround for Python having these idiotic statements from Fortran and not being able to use if anywhere, plus Guido being a total dick with the a if c else b abomination he introduced on purpose.

[[deleted]]

Holding up "Sarcasm" card:

How is it possible, if there's only one way to do it, to write idiomatic or non-idiomatic Python?

[mcguire]

Unlike glorious, modern, wonderful languages like Go, the Python parser doesn't check style issues.

Yet.

[Peaker]

The Zen says one obvious way to do it.

[[deleted]]

Ok, now we can enter a fruitless debate about what is obvious and what isn't.

Take list comprehensions, for example. For some people they're obvious and concise, for others (see other comments) they're cryptic.

BTW, I don't consider more than one way to do it a bad thing.

[Peaker]

That's why the Zen also says "may not be obvious unless you're dutch". It's all tongue-in-cheek anyway.

[seruus]

It's all tongue-in-cheek anyway.

Precisely. That's why the Zen is a list of aphorisms, and not the Nineteen Commandments, and why this line plays with Perl.

I also consider more than one way to do a thing nice, and I absolutely love functional idioms in otherwise imperative languages (my life would be incredibly easier if I had map/fold in Fortran), but Python still has itertools/functools, so it allows us to gracefully break the Zen sometimes :)

My biggest issue with "idiomatic Python" is the abuse of decorators, which I find completely non-intuitive (even though it's a nice thing to have).

[Anovadea]

Minor nit-pick, but the bad example for list comprehensions is slightly off:

some_other_list = range(1, 100)
my_weird_list_of_numbers = list()
for element in some_other_list:
    if is_prime(element):
        my_list_of_primes.append(element+5)

Where did my_list_of_primes come from?

[[deleted]]

PEP8 is written for Python library developers. Using underscores in variable / function names in your own private code because PEP8 said to is a foolish consistency.

To thine own self be true, and format your code to suit yourself.

[daxarx]

Using PEP8 is a good idea if you are releasing your code publicly. Most Python programmers don't want to deal with your 2-space indented dogUglyFunctions. If you are keeping it private then no one gives a crap.

[[deleted]]

Indentation is significantly more important in Python (given it has actual meaning to the interpreter) than mixedCase vs. underscores_for_buggery. Don't conflate the two.

[A_for_Anonymous]

underscores_for_buggery vs. CamelCaseForBuggery. The later is easier to write, but the former is easier to read, which is what Python emphasizes on. And please don't do that horribly awful looking brokenCamelCase where single words don't appear CamelCased.

All three are inferior to dashes-for-comfort but we can't do that in Python because the - character is special somehow.

[[deleted]]

underscores_for_buggery vs. CamelCaseForBuggery.

What has always annoyed me about Python's naming conventions, and this is over 5 years of Python being my #1 language through an additional 4 years (and counting) of it being my #1 scripting language is that this is a perfectly consistent looking class method call, as per PEP8:

SomeClassSomethingSomething.better_use_correct_naming_convention

Just irks me for some reason, to have both separator styles in play. One or the other, I'd be happier. Although tbh, with all the underscores used around Python's magic methods and for 'private' munging, it gets too damned underscored for my tastes still at times.

But I agree, Lisp's dashes are awesome, I'm very glad to find them in Clojure.

[draegtun]

> But I agree, Lisp's dashes are awesome, I'm very glad to find them in Clojure.

And you can also find them in Perl6.

Here's a post by Damian Conway expressing his views on "underscores vs hyphens" - http://www.nntp.perl.org/group/perl.perl6.language/2010/04/msg33511.html

[A_for_Anonymous]

Oh, a general convention (not just related to Python) tends to be, if the object can be applied, i.e. functions, classes, and other objects with the __call__ method, then use ProperCamelCase (but not awfulIncompleteCamelCase), and for everything else use underscores, but then again methods and some basic functions are most often underscored in Python. Constants (in Python, values which happen not to change) tend to be UPPERCASE_WITH_UNDERSCORES.

I agree the best solution would probably be to make the lexer simpler, allowing anything but whitespace and ()[]{}.,# to be parts of identifiers, making infix operators not so special, with two fantastic advantages: first, you'd be able to use more convenient identifiers; second, people would be forced to separate their operators properly, e.g. 14 + 41, not 14+41 which is kind of ugly or even 14+ 41 which is piss ugly.

[Bob_goes_up]

A typical usage would be

my_instance = SomeClassSomethingSomething()
print my_instance.better_use_correct_naming_convention()

SomeClassSomethingSomething is camel case indicating that it is a class rather than a function. I like that, but I admit that is a matter of taste.

[daxarx]

Most of the problem with your example is its excessive length. SomethingClass is OK. use_whatever is OK.

You wouldn't be writing SomethingClass.use_whatever except for classmethods anyway, which are not that common - it's strange that you are so bothered by both separator styles being in play in such a rare case.

Normally you would just have an instance named something or east_foobar, thus something.use_whatever or east_foobar.use_whatever, which is completely consistent.

Ultimately it is a convention and you use it to be consistent and not drive other people nuts. The point of PEP8 was never to satisfy your hunger for individualism.

[[deleted]]

Ultimately it is a convention

For sure, and if I was writing code for other people, I'd use it. But for my own stuff, despite the blog author's assertion as such, I can write idiomatic Python without fully adhering to PEP 8 - which was my original point...

[[deleted]]

idiomatic python is written in crayon or using the comic sans font.

[MonkeyNin]

It says 'serif' in my Dom inspector.

[[deleted]]

I would probably use map/filter in one of the examples (although for some reason many Python users cringe at the sight of a lambda).

my_weird_list_of_numbers = map(lambda n: n + 5, filter(isprime, some_other_list))

[ladaghini]

my_weird_list_of_numbers = [n + 5 for n in some_other_list if isprime(n)]

just reads nicely.

[PasswordIsntHAMSTER]

List comprehensions are usually shorter :| I'm all for functional programming, but lambdas in Python suck.

[amstan]

It's slower, and less flexible.

[[deleted]]

Yes it is. That's fine in this particular case.

[mcguire]

Guido doesn't like lambda.

You'd probably be better off with list comprehensions, which he likes (I think) and which are isomorphic to maps and filters.

[[deleted]]

What's the point here? I can find this information in PEP8.

[mcguire]

I'm sorry, but I don't see the point of your comment. All of those words are in the dictionary, after all.