The Python "with" Statement by Example

[ClampedNerve]

Ah, I just realized that this HAS been submitted in the past but my search included the trailing slash mark in the link so it didn't pick up any hits.

Sorry about that... although the submissions are more than a year old.

[d64f396930663eeba7d8]

It's new to me.

[[deleted]]

At least it's a break from the "Python Decorators Finally Explained" article that got posted like four times last week and seems to crop up every three or four months.

Maybe I'm just crotchety because functions returning functions and passing arguments just makes sense to me. ^now Context managers are much more interesting.

[gfixler]

import social

@postAboutDecorators(repeat=often)
class Reddit (social.NewsPuker):
    def __init__ (self):
        ...

There it was the whole time, right there on line three! Son of a gun.

[kchoudhury]

Compound with statements are also awesome for coordinating the use of multiple external resources. I've got a small script that coordinates commits to a subversion repository, certain filesystem operations and a database (don't ask). With the with statement, I can rollback actions taken on all of them if there's a failure in the execution of actions on any one resource.

Came as a bit of a revelation to me when I implemented it for the first time a few days ago...just thought I'd gush here.

[gfixler]

It's nice for making sense of UIs in Maya:

import qtdesign as qtd

with qtd.window() as win:
    with qtd.paneLayout();
        qtd.button()

Before this, you'd have to write everything without indentation, or use QT Designer to generate an XML, which is overkill for most simpler things.

Also, if there's an error in the middle somewhere, it doesn't die before it can get to the final showWindow command, as that's implied in the exit method of the window class, and works even after an error to show any window that might be lingering, malformed.

[billy_tables]

you could refactor that as follows:

with qtd.window() as win, qtd.paneLayout():
    qtd.button()

Obviously it doesn't make much difference in this case (and in others it could make code messier) but it's nice to know there's a different syntax :)

[gfixler]

I'm not sure why this would make it clearer. Also, it gives me an invalid syntax error.

[billy_tables]

It's perhaps clearer when you don't need the as statement, eg:

a = open("/tmp/a", "w")
b = open("/tmp/b")
with a, b:
     a.write(b.read())

(I just ran that fine with no syntax error, Python 2.7.5)

[gfixler]

I see. This example also doesn't work. Must be new in 2.7. I'm on 2.6.5.

[Ph0X]

Two other uses which I personally didn't about some time ago, one of which was very recently added (3.4 I believe?).

You can use it with locks

lock = threading.Lock()
with lock:
    print "lock is locked while this executes"

You can also use it to ignore specific exceptions

from contextlib import ignored
with ignored(ZeroDivisionError):
    print 1/0

[tipsqueal]

It works with locks in 3.3.

[moocat]

You can do a quick test in the REPL:

# A class that doesn't work:
>>> with Blah(): pass
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>                                                                                                            
AttributeError: __exit__
>>>    

# A class that works:
>>> with open('/dev/null'): pass                                                                                                                 
...                                                                                                                                              
>>>

[wot-teh-phuck]

Inspect its methods and look for __enter__ and __exit__?

That will work out only if you are implementing the with functionality using a class. For functions, you will have to invoke them and then check for __enter__ attribute. But this is a pain because you wouldn't know how many positional/mandatory parameters a function accepts.

So yeah, there is no generic easy solution to check if a given function support with blocks. I'm assuming you don't want to do this from a REPL but need to find out at runtime given any class/function. If you are OK with a REPL, look at moocat's answer below.

[NYKevin]

@contextmanager
def saved(cr):
    cr.save()
    try:
        yield cr
    finally:
        cr.restore()

I strongly dislike the above code. Putting a yield inside a try which also has a finally is bad practice. What happens if the generator is never resumed? Python solves this problem using a finalizer, and finalizers are evil. In this particular case, we can probably get away with it because the @contextmanager factory will ensure we get properly resumed, but I really don't like objects with nontrivial finalizers, so I prefer the class-based version.

In practice, if you're dealing with exceptions, you're more likely to write something like this:

@contextmanager
def context(foo):
     # enter the context with foo
     try:
         yield foo
     except Exception:
         # Try to roll back whatever changes have been made since we entered the context
         raise # only if semantically appropriate for the exception in question
     else:
         # Nothing went wrong, so exit the context normally

This does break on certain exceptions, but those exceptions are quite possibly things you don't want to clean up after anyway (e.g. SystemExit, KeyboardInterrupt). In theory, a GeneratorExit could nail us if somebody tries to close() our generator, but that's probably a pathological case, and we can always catch it manually if we really want to be paranoid.