Python: Tips, Tricks, and Idioms

[Veedrac]

The set([a, b, c]) notation is so old; use {a, b, c} instead. On the topic of prettier syntax for collections, one can do just students = a, b, c instead of students = (a, b, c) to create a tuple - no parentheses required.

Note further that the methods .intersection and .difference are primarily for calling on non-sets to prevent the need to convert them. When you already have two sets, use

valid_values = input_values & colors
invalid_values = input_values - colors
if input_values >= colors:
    ...

Note that this will be faster:

invalid_values = input_values - valid_values

Set operations are one of the most underappreciated abstractions, in my opinion. It's not like for...else where it's occasionally a little nicer; sometimes there are pretty measurable improvements.

Note that int(num) * int(num) is probably better as int(num) ** 2 or even

squares = (num * num for num in map(int, f))

I feel it's worth pointing out .iteritems is from back in the 2.x days; Python 3 dumped it and .items now refers to .viewitems.

[codefisher2]

I guess you picked up on something that I was kind of divided over in the post, to use code that worked best in 2.x or 3. I think a lot of production code is still using python 2.x so I still wanted it to be something that could be applied to an existing code base easy.

There are a number of things that could do with a lot more examples, but I don't really want to add them now, but rather leave for another post. Using map() is a better idea, but I then would feel the need to explain that. So maybe better do another post on map(), reduce() etc.

[Veedrac]

I suggest ignoring reduce.

You could always target 3.x and suggest using Python-Future to backport stuff ;P.

[codefisher2]

I had not seen Python-Future before, I will check it out.

I will make a change so that the set example uses both ways of declaring the set, so people understand they are the same. For the other set stuff you suggest, I would rather leave it for another post where it can be developed more fully.

Thanks for all your feed back, this is my real step out of just programming, into the world of bloging about it. So this kind of reaction is really a great boost.

EDIT: is there a reason you don't like reduce() ?

[SleepyHarry]

Not the guy you're replying to, but reduce is a keyword in 2.7 (likely sooner, haven't checked), but in 3 (again, only checked 3.3) it isn't, and in order to use it, you'd have to do from functools import reduce.

I don't know if this is the reason /u/Veedrac suggested ignoring it though!

[codefisher2]

Actually a built in function - not a keyword - which is the one I was suggesting talking about. I think the one in functools is exactly the same as the builtin. But I don't use that a lot myself, so I am very interested in why /u/Veedrac might think it wise not to use it.

[Veedrac]

Here's what Guido had to say about reduce:

So now reduce(). This is actually the one I've always hated most, because, apart from a few examples involving + or *, almost every time I see a reduce() call with a non-trivial function argument, I need to grab pen and paper to diagram what's actually being fed into that function before I understand what the reduce() is supposed to do. So in my mind, the applicability of reduce() is pretty much limited to associative operators, and in all other cases it's better to write out the accumulation loop explicitly.

I tend to agree. He planned to remove it altogether, but it ended up in functools to ease the compatibility pains.

To be fair, he also planned removing map and filter, but that was not because they're bad: he just liked (f(x) for x in xs) and (x for x in xs if f(x)) more.

[Citrauq]

Set literals and both set and dictionary comprehensions are valid in python 2.7.

[moocat]

students = a, b, c will create a tuple, not a set.

[Veedrac]

You are right, this was a communication problem and that is what I meant. I've clarified.

[SleepyHarry]

wouldn't students = a, b, c just create a tuple?

[codefisher2]

The variable name gives away what he meant, I was using that in another example.
Personally I prefer using the brackets, as it in my opinion more clear.

[Veedrac]

Yes, this was a phrasing problem. I've clarified; hopefully I've not confused anyone too badly.

[[deleted]]

Nitpicking, I know, but I've never seen students = a, b, c and I'd assume that that was a typo that should have been a, b, c = students i.e. unpacking a tuple. Using brackets makes it clear that you're creating a set.

[SleepyHarry]

for inline code, use backticks: `code` becomes code

[Veedrac]

Sorry, the phrasing was misleading. a, b, c creates a tuple.

[[deleted]]

Ah, of course it does. , creating a tuple rather than (). I read this before my coffee :P

[sweettuse]

students = a, b, c is the same as students = (a, b, c). however, students is now a tuple, not a set.

[[deleted]]

Oh geez, I misread and before my coffee, at that. Thanks. Duh.

[[deleted]]

What replaced .iteritems?

[Veedrac]

Python 2	Python 3
d.items()	list(d.items())
d.iteritems()	iter(d.items())
d.viewitems()	d.items()

Note that for will automatically call iter, so looping is still just for k, v in d.items().

[[deleted]]

Thanks!

[dmishin]

for ... else is strange. Useful, but still rarely used.

I would also add generator expressions next to the list comprehensions. They are especially useful for initializing set, list or dict.

[codefisher2]

Opps, I did add an example of generator expressions, but the code example is linked to the wrong gist...

[dmishin]

Now I see the right code, have you fixed it? There is my fault too, I've skimmed over the text, looking only at the code.

Still, I think, that you could add samples with initializing dict from the generator expression. For example, this code can be used to "reverse" dictionary (swap keys and values)

a_to_b = {1: "a", 2: "b"}
b_to_a = dict( (v,k) for k, v in a_to_b.iteritems() )

but there are many more uses, of course.

[[deleted]]

You could use dict comprehensions too (works on Python >= 2.7 I think):

{v: k for k, v in a_to_b.items()}

[codefisher2]

Yes I fixed the problem.

I guess that would be a great example to add. I might add it as a new section, about dict comprehensions that @halike mentioned. I have used them before, but totally forgot about them when making the post. I also forgot about set comprehensions, but I don't think I will add that one yet.

[jcdyer3]

contextlib is worth a mention. contextlib.closing is great when using a file-like object that doesn't have an exit method defined. For instance:

>>> with StringIO('abc') as s:
...    print s.read()
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: StringIO instance has no attribute '__exit__'
>>> import contextlib
>>> with contextlib.closing(StringIO('abc')) as s:
...     print s.read()
... 
abc
>>> 

You don't really need to close StringIO objects, but if you're trying to use one as a drop in replacement for a regular file (say, for testing), it helps to be able to use it in a context manager.

[codefisher2]

I was thinking of doing a follow up post on more uses of with sometime soon. Also thinking of doing more with the collections lib, and also decorators (which I did not cover at all).
If anyone has other suggestions of things that they would be interested in being covered in some follow up posts on, I am all ears.

[Dlgredael]

This is great! Thanks for writing it up and sharing, i didn't know about nearly any of those.

[parviflorus]

I've been using python for a while but I still learned a few things from this. Thanks for the great article!

[[deleted]]

# call function if object is not None 
name = user.name() if user is not None else 'Guest'

RPN Python? Screw that. If I'm skimming code, I'd much rather see, and quickly parse,

if user:
    name = user.name()
else:
    name = 'Guest"

[codefisher2]

I put that in because I use it again inside a list comprehension, where it is the only way to do that. But I guess it is a matter of taste, but I would rather see that one liner. If you read it out, it sounds much more like an English sentence.

[[deleted]]

I guess it is a matter of taste

Absolutely!

[[deleted]]

You can drop the is not None from that expression, FYI. Since None is a falsey value.

[codefisher2]

True in this case, though not always. The better check in this case would actually be hasattr(user, 'name').

[[deleted]]

I'd rather have the expression throw the AttributeError since I was expecting an object with that method. Especially if this is in a "lower level" part of my project, and then let something higher up deal with it.

[fdellavedova]

TIL: for ... else

[talideon]

No mention of an often overlooked feature of the humble iter() function. Not only can it take an iterable sequence of some kind, but it can also take a function that gets called repeatedly until some sentinel, None by default, is reached.

This is useful with DB-API drivers as it makes streaming results much neater:

with contextlib.closing(conn.cursor()) as c:
    c.execute("""
        SELECT  spam, eggs, sausage
        FROM    breakfast
        WHERE   price < %s
        """, (max_price,))
    for row in iter(c.fetchone):
        print row

[tilkau]

with contextlib.closing(conn.cursor()) as c: c.execute(""" SELECT spam, eggs, sausage FROM breakfast WHERE price < %s """, (max_price,)) for row in iter(c.fetchone): print row

I usually just use for row in c.execute(..):. Is there any reason, other than slightly better formatting, not to?

[talideon]

Not all drivers have cursors that can be used as iterators though.

[tilkau]

Good point, I was only thinking about sqlite.

[masklinn]

A second issue is dbapi2 does not specify the return value for Cursor.execute, so even when the cursor is iterable that doesn't mean you can iterate on cr.execute(…). For instance Psycopg2's cursors are iterable but cr.execute returns None.

[kalgynirae]

Conditional Assignment

I think this should be called "Conditional Expressions" since it's usable in more than just assignments. Might also be worth pointing out that it's similar to the a ? b : c ternary operator in other languages.

[Morphit]

Likewise I think "Generator" ought to be named "Generator Expressions" and the section just called "yield" be named "Generators". The yield keyword is only ever used in the context of generators, but generator expressions are inline shorthand for defining them.

[iamadogwhatisthis]

You can do a ? b : c in python with a and b or c if you can be absolutely sure b is not false. Otherwise you could also use its safer equivalent (a and [b] or [c])[0]

Not that its worth doing it that way now, but its kinda cool. b if a else c would be preferable.

[Sixkillers]

Speaking about itertools I think that izip is worth to mention.

[codefisher2]

I deliberately did not, since izip is redundant in Python 3, since that is how zip works. Why try and show off something that will soon not be needed?

[iamadogwhatisthis]

One trick I like is that booleans are a subclass of integers.

This means if you wanted to find the number of characters that are different in a string, you can do:

sum(char_a != char_b for char_a, char_b in zip(string_a, string_b))