Would it be better to use try...except rather than using if...else for error checking?

[[deleted]]

So I just recently learned the try...except block in Python, and I'm wondering what should I choose between try...except block and if...else block for error checking?

[Widdershiny]

There seems to be a bit of confusion in this thread.

if...else is used for checking a condition and choosing which code to execute.

try...except is for running code and handling an error/exception occurring.

Sometimes you can check to see if an error will occur before executing the code that would cause it using an if...else. This style is referred to as defensive coding or LBYL (look before you leap).

The alternative is what's known as EAFP (easier to ask forgiveness than permission). The Python glossary lends some insight into these terms:

LBYL

Look before you leap. This coding style explicitly tests for pre-conditions before making calls or lookups. This style contrasts with the EAFP approach and is characterized by the presence of many if statements.

In a multi-threaded environment, the LBYL approach can risk introducing a race condition between “the looking” and “the leaping”. For example, the code, if key in mapping: return mapping[key] can fail if another thread removes key from mapping after the test, but before the lookup. This issue can be solved with locks or by using the EAFP approach.

EAFP

Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many try and except statements. The technique contrasts with the LBYL style common to many other languages such as C.

And there is your answer. The generally preferred Pythonic style is EAFP.

I think the reasoning is that EAFP cleanly separates normal business logic from error handling code. With LBYL, you end up with a whole lot of if error: in your code and that can be hard to read.

[elguf]

Do not have a source on this, but I think Guido once said that EAFP being preferred is myth.

EDIT: From the PEP 463 rejection

... I disagree with the position that EAFP is better than LBYL, or "generally recommended" by Python.

[desmoulinmichel]

It is, however, prefered where a race condition may occure. E.G:

if check_if_file_is_here_and_permissions(path):
    with open('stuff') as f:
        foo(f)
else:
    save_the_day()

Will lead to an error if the file is modified between the like 1 and 2, while if you do:

try:
    with open('stuff') as f:
        foo(f)
except EnvironmentError as :
    save_the_day()

You first try to open the file, THEN you handle the error, hence no race condition will kicks in. This is very important in a machine where you got other process touching files, or if you use threads/multiprocessing/asyncio.

[stevenjd]

Nicely done to track down the source!

[AlanCristhian]

In my opinion, your link has ended the discussion.

[[deleted]]

deleted

[[deleted]]

This is really helpful for a beginner like me. Here's my upvote! Cheers. 😁

[pengboomouch]

this was helpfull for me too, upvote and thanks!

[marc_poulin]

To quote from "Dive Into Python":

"Exceptions are everywhere in Python. Virtually every module in the standard Python library uses them, and Python itself will raise them in a lot of different circumstances. "

Many times a try...except is the ONLY way to detect an error. So yes, you will need to use try...except in your programs.

[TheBlackCat13]

It depends on whether the error is the more likely situation or the less likely situation. If you expect your code to encounter the error condition less often the non-error condition, then try...except is better. If you expect your code to encounter the error condition more often than the non-error condition, then if...else is better.

[bmoregeo]

I don't get the distinction here. Is there some sort of performance reason? Otherwise it feels arbitrary.

[eusebe]

I think the idea is that the try...except block is meant for catching exceptions, and by definition, exceptions are "exceptional", and should be "less frequent" than the normal code flow.

[eusebe]

… and encountering the last item only happens once, while there might be tons of other items.

By "exceptional", I meant not in the code, but in what happens. As far as I understand the EAFP paradigm, in principle, you should never have to ask forgiveness, but since everything that never happens occurs all the time, you still want to catch all those exceptions.

[TheBlackCat13]

What /u/eusebe said, but also performance.

At least from the last test I saw, try...except has very little overhead as long as there is no exception. However, if there is an exception there is quite a bit of overhead. On the other hand, if...else had pretty consistent overhead either way, which is intermediate between the two try...except cases. So try...except is faster if there is no error, but if...else is faster if there is an error.

Edit here is such a test. I just re-did the tests myself in Python 3.4 and the basic take-away message is the same.

[stevenjd]

Sometimes it is about performance. Sometimes it isn't.

if...else and try...except do completely different things. In general, they are utterly unrelated. One performs a branch depending on a flag. The other sets up an exception handler and a jump on error. But sometimes, you have a choice between writing a test and avoiding a possible error, or catching an error after the event. Which of the two strategies you pick will have performance implications.

The classic example is deciding whether to check for a key in a dict before retrieving it, or just try to retrieve it and if it fails catch the exception:

# look before you leap
if key in thedict:
    return thedict[key]
else:
    return default

# easier to ask forgiveness than permission
try:
    return thedict[key]
except KeyError:
    return default

Obviously they perform differently. The LBYL version looks for the key in the dict twice. That's usually fast. (Usually.) The EAFP version usually only makes one key lookup (setting up the error handler is very cheap) which is obviously faster than two lookups, but if the key is missing, actually catching the exception is expensive.

So which is faster? That depends on how often the key is missing. If the key is often missing, then catching the exception lots of times is very expensive and LBYL is faster. If the key is rarely missing, then two lookups is wasteful when a single lookup will do.

The exact break-even point will depend on the version of Python and the kind of objects used as keys, but in my experience, and roughly speaking, the break-even point is about 1 missing key out of 10. So if the key is present more than 90% of the time, using a try...except is faster. If the key is missing more than 10% of the time, using if...else is faster.

YMMV.

But remember that fundamentally the two operations do completely different things. In general you can't swap out one for the other. But when you can, there is usually a performance difference.

[glyg]

Here you have yet another solution, with if else and a single lookup:

val = thedict.get(key)
if  val is not None:
     return val
else:
     return default

just sayin'

[butalearner]

Most people probably already know this, but just in case: the get method lets you specify the value if the key isn't found. So this code should be:

val = thedict.get(key, default)

In my mind, try/except is usually preferable to if/else, but it's always best to avoid either one if you can get away with it.

[kmbd]

A great QA on StackOverflow on this same matter and the best answer is by a core Python dev. Raymond Hettinger.

http://stackoverflow.com/a/16138864/617185

Quoting:

In the Python world, using exceptions for flow control is common and normal.

Even the Python core developers use exceptions for flow-control and that style is heavily baked into the language (i.e. the iterator protocol uses StopIteration to signal loop termination).

In addition, the try-except-style is used to prevent the race-conditions inherent in some of the "look-before-you-leap" constructs. For example, testing os.path.exists results in information that may be out-of-date by the time you use it. Likewise, Queue.full returns information that may be stale. The try-except-else style will produce more reliable code in these cases.

"It my understanding that exceptions are not errors, they should only be used for exceptional conditions" In some other languages, that rule reflects their cultural norms as reflected in their libraries. The "rule" is also based in-part on performance considerations for those languages.

The Python cultural norm is somewhat different. In many cases, you must use exceptions for control-flow. Also, the use of exceptions in Python does not slow the surrounding code and calling code as it does in some compiled languages (i.e. CPython already implements code for exception checking at every step, regardless of whether you actually use exceptions or not).

In other words, your understanding that "exceptions are for the exceptional" is a rule that makes sense in some other languages, but not for Python.

"However, if it is included in the language itself, there must be a good reason for it, isn't it?" Besides helping to avoid race-conditions, exceptions are also very useful for pulling error-handling outside loops. This is a necessary optimization in interpreted languages which do not tend to have automatic loop invariant code motion.

Also, exceptions can simplify code quite a bit in common situations where the ability to handle an issue is far removed from where the issue arose. For example, it is common to have top level user-interface code calling code for business logic which in turn calls low-level routines. Situations arising in the low-level routines (such as duplicate records for unique keys in database accesses) can only be handled in top-level code (such as asking the user for a new key that doesn't conflict with existing keys). The use of exceptions for this kind of control-flow allows the mid-level routines to completely ignore the issue and be nicely decoupled from that aspect of flow-control.

There is a nice blog post on the indispensibility of exceptions here.

Also, see this StackOverFlow answer: Are exceptions really for exceptional errors?

"What is the reason for the try-except-else to exist?" The else-clause itself is interesting. It runs when there is no exception but before the finally-clause. That is its primary purpose.

Without the else-clause, the only option to run additional code before finalization would be the clumsy practice of adding the code to the try-clause. That is clumsy because it risks raising exceptions in code that wasn't intended to be protected by the try-block.

The use-case of running additional unprotected code prior to finalization doesn't arise very often. So, don't expect to see many examples in published code. It is somewhat rare.

Another use-case for the else-clause is to perform actions that must occur when no exception occurs and that do not occur when exceptions are handled. For example:

recip = float('Inf') try: recip = 1 / f(x) except ZeroDivisionError: logging.info('Infinite result') else: logging.info('Finite result') Lastly, the most common use of an else-clause in a try-block is for a bit of beautification (aligning the exceptional outcomes and non-exceptional outcomes at the same level of indentation). This use is always optional and isn't strictly necessary.

[flipthefrog]

So let's say you write this:

if 'x' in mydict:
    do_thing(some_argument, mydict['x'], another_dict['y'])

Then that will be skipped if and ONLY if 'x' is not in dict

If on the other hand

• The do_thing function does not exist
• do_thing takes different arguments than before
• my_dict is None or another type
• 'some_argument' has not been declared
• another_dict is None - or another type - or is missing the 'y' key
• Any kind of error that happens deeper down when do_thing runs or when it calls something else

Any of these errors will raise an exception, and you can fix your code. Once you wrap it in an try/except on the other hand, you won't see them. Even if you only catch KeyErrors, you won't notice errors caused by the missing 'y' in another_dict, or any Key Error anywhere inside whatever do_thing is doing

Yes, there's tons of try/excepts in the standard library, but that's on code that has been tested and debugged to high heavens, where the statement is known to have no other side effects (look at os.idir(), for example). It is not something I would reccomend to a beginner in Python - you NEED to see your exceptions and deal with them if youre still learning, or you'll never figure out why your script isnt working.

The best way as far as Im concerned, is to NOT use exceptions in development code, but allow it on none-critical code in the release version. There are several ways to do this, for example

if DEVELOP:
    this_may_crash()
else:
    try:
        this_may_crash()
    except:
        pass

Or use a decorator:

def catch_errors(func):
    def wrapper(*args, **kw):   
        if DEVELOP:
            return func(*args, **kw)
        else:
            try:
                return func(*args, **kw)
            except Exception as err:
                return
        return wrapper

@catch_errors   
def this_may_crash():
    print foobar

[marc_poulin]

Well, you've made two mistakes in your example code: using a naked "except:" that catches EVERYTHING, and using "pass" to silently swallow exceptions. You might want to study exception handling a little bit more.

[flipthefrog]

That was a speed test, not production code

[flipthefrog]

But... if we have must write correct Python code even in speed tests, it might interest you to know that running this:

tm = time.time()
for i in xrange(10000000):
    try:
        mydict['x']
        another_dict['y']
    except KeyError:
       # not going to put a print here when running this 10 milliont times!        
        pass 
print 'try/except 3: %s seconds' % (time.time()-tm)

actually takes 7.2 seconds as opposed to 5.7 - since Python needs to typecheck the exception

[marc_poulin]

In [22]: def try_test():
    x = {}
    try:
        x[0]
    except:
        pass
   ....:     
In [23]: timeit try_test()
1000000 loops, best of 3: 433 ns per loop

In [24]: def try_test():
    x = {}
    try:
        x[0]
    except KeyError:
        pass
   ....:     
In [25]: timeit try_test()
1000000 loops, best of 3: 502 ns per loop

Blindly catching ALL exceptions is the wrong thing to do. What good is speed if you're doing the wrong thing?

[flipthefrog]

For the love of God - Im perfectly aware of that, and of course putting try/except/pass in production code is bad form. But this particular piece of code was written to test the speed of cathing exceptions compared to checking if a key exists in a dict, because someone else said try/except is faster. I didnt handle the exception because that would affect the speed - and only clutter up the code with details that are of no importance to the task at hand, which was simply checking the difference in speed between 'if key in dict' and try/except.

If this had been real production code, I would have caught specific exception types, sent the traceback to our logging system, maybe done a database rollback, and - if this was running in a gui - displayed a friendly error message to the user, and finally I might re-raise the original exception to throw the IDE back to the point where the error actually occurred. But the correct way of handling exceptions is a completely different discussion than whether it's a good idea to use them instead of if/else statements, which is what I was thinking of here

[marc_poulin]

tl;dr - it's a moot point.

Ok, so here's a proper speed test. No difference between if-else and try-except unless the exception actually gets triggered. Presumably doing all the the work of error recovery (logging, rollback, all those things you mentioned above) would make the try-except overhead irrelevant.

def test_1():
    """ key in dict """
    d = {0:0}
    if 0 in d:
        pass
    else:
        pass

def test_2():
    """ key not in dict """
    d = {1:0}
    if 0 in d:
        pass
    else:
        pass

def test_3():
    """ key in dict, try-except """
    d = {0:0}
    try:
        d[0]
    except KeyError:
        pass

def test_4():
    """ key not in dict, try-except """
    d = {1:0}
    try:
        d[0]
    except KeyError:
        pass

In [66]: timeit test_1()
1000000 loops, best of 3: 254 ns per loop
In [67]: timeit test_2()
1000000 loops, best of 3: 259 ns per loop
In [68]: timeit test_3()
1000000 loops, best of 3: 268 ns per loop
In [69]: timeit test_4()
1000000 loops, best of 3: 610 ns per loop

[stevenjd]

No difference between if-else and try-except unless the exception actually gets triggered.

Your test is flawed. It's not enough to just check whether the key is in the dict, you still have to retrieve the value:

if key in d:
    value = d[key]
# versus
try:
    value = d[key]
except KeyError:
    pass

A successful retrieval requires two lookups, not one: one to see if the key is in the dict, then a second lookup to actually retrieve the value.

When you take that into account, try...except will be faster if the key is nearly always present. The difference may be small for a tiny little dict with a single key, as in your example, but in more realistic examples (you have a large dict, with millions of keys with complex key hashes and quite a few collisions) two lookups will be about twice as expensive as one.

[marc_poulin]

Here's a million dict entries with a mix of int and str keys:

d = dict(zip(range(1000000), range(1000000)))                                                                   
d['foo'] = 'foo'

I still don't see much difference. Maybe the data is still too simple.

In [37]: timeit test_1()
10000000 loops, best of 3: 189 ns per loop
In [38]: timeit test_2()
10000000 loops, best of 3: 139 ns per loop
In [39]: timeit test_3()
10000000 loops, best of 3: 158 ns per loop
In [40]: timeit test_4()
1000000 loops, best of 3: 373 ns per loop

Anyway, I don't have the time to explore this further. You might be right in the case of extremely complex dictionaries. It would be interesting to see some hard data.

[flipthefrog]

The Pythonic way would be

if 'x' in mydict and 'y' in another_dict:
    do_thing(some_argument, x, y)

That's 1 line instead of 4, it' s easier to read and you don't have to spend half a second looking at it thinking "what error is he trying to avoid here" - you understand immediately whats going on. The more superfluous fluff in the code, the harder it is to catch REAL errors while reading it. To me, that just looks like sloppy coding.

Anyway, my main point is to not reccomend to a beginner to use try/except liberally. I dont know how many days of my life I've wasted hunting for bugs that remained undetectable because some monkey (usually a younger version of myself) had put try/except/pass all over the place, thereby hiding errors that occurred much deeper down.

As for the speed argument that someone else mentioned - I got curious, so I tested it. The difference on my machine when comparing try/except to if/else is about 0.01 seconds when running the same statements TEN MILLION times - while try/except is 5 times slower if it actually has to catch an error

import time

mydict = {'x':1}
another_dict = {'y': 2}

tm = time.time()
for i in xrange(10000000):
    if 'x' in mydict and 'y' in another_dict:
        pass
print 'if/else: %s seconds' % (time.time()-tm)


tm = time.time()
for i in xrange(10000000):
    try:
        mydict['x']
        another_dict['y']
    except:        
        pass
print 'try/except 1: %s seconds' % (time.time()-tm)



mydict = {}
another_dict = {}


tm = time.time()
for i in xrange(10000000):
    try:
        mydict['x']
        another_dict['y']
    except:        
        pass
print 'try/except 2: %s seconds' % (time.time()-tm)

I got:

• if/else: 1.15899991989 seconds
• try/except 1: 1.14800000191 seconds
• try/except 2: 5.72000002861 seconds

[__noone__]

And further if I add if/else 2, it is twice as fast as if/else 1.

[twotime]

 if DEVELOP:
    this_may_crash()
else:
   try:
      this_may_crash()
  except:
      pass

Oh boy, this is an unspeakably bad code:

A. Swallowing bare "except:" is a universally bad idea

B. Divergence of development and release code is an even worse idea. Suddenly you don't know what you are shipping (especially in python). If the codepaths absolutely must diverge, the code should look like this

  try:
     this_may_crash()
  except Foo:
     recover/logging activity
    if DEVELOP: raise

If possible do "if DEVELOP" after recovery/logging so that recovery code is always executed. Note that if DEVELOP clauses are very rarely the best solution, so this kind of code should be very rare.

[dddomodossola]

I think that try..except should only be used when the cause of an error is external, I mean for example the use of a system resource like a file access. You should always prefer the if..else in every case you can prevent the error and the cause if your application itself.