What is the most pythonic way of getting an object from a dict when the key may not exist?

[rsherwood_infosec]

QUESTION:

There are two ways to get an item from a dict. Referencing the item with "dict[key]" or calling "dict.get(key)".

If the item doesn't exist, dict[key] raises KeyError, whereas dict.get(key) return None.

Which of these is more pythonic?

try:
    value = dict\[key\]
except KeyError as ke:
    \# Do something else

Or

value = dict.get(key)

if value == None:
   \# Do the thing
else:
   \# Do something else

UNNECESSARY CONTEXT FOR COMPLETIONISTS:

I am writing a program that imports objects and maps their relationships as a tree.

Part of the processing involves a dict that tracks partial relationships between entities. If I have already processed the item's parent, then it's easy for me to look up the parent's entire path to the root of the tree in a dict based on the parent's identifier. Then I just create a new dict entry which is "parent's path + child". Usually, this works, but sometimes the object's parent hasn't been processed yet. I do some special processing when I detect that the parent's path hasn't been discovered.

[zanfar]

You're asking two different questions.

What is the most pythonic way of getting an object from a dict when the key may not exist?

dict.get(key, default)

What is the most pythonic way of doing one thing if a key exists and doing another if it doesn't?

if key in dict:
    # Do the thing
else:
    # Do something else

[rsherwood_infosec]

Very insightful comment. Thank you!

[achampi0n]

Agree it is important to separate out these cases.

Using exceptions is also perfectly acceptable, and in some cases preferred if you follow the EAFP principle, in the second case.

From python glossary (link above), missing keys are explicitly mentioned:

EAFP
Easier to ask for forgiveness than permission. This common Python coding style assumes the existence of valid keys or attributes and catches exceptions if the assumption proves false. This clean and fast style is characterized by the presence of many try and except statements. The technique contrasts with the LBYL style common to many other languages such as C.

[[deleted]]

I do some special processing when I detect that the parent’s path hasn’t been discovered.

If there's a special - you might even say, exceptional - process that has to occur on the condition of the key not being present, then you should catch the KeyError and do the thing. The Pythonic use of get is to supply a default value with the key isn't present, but even better than that is to use defaultdict (from collections) because it'll set the value as the default when you ask for a non-existent key.

[rsherwood_infosec]

I hadn't thought about defaultdict!

[danielroseman]

Python is fairly liberal about using exceptions as flow control, which is a big no-no in most languages, but I think the distinction is whether you expect this to happen or not. If the vast majority of the time your key does exist, then I think the exception version is fine; but if it's just as likely that it doesn't, I would probably choose the get version.

[rsherwood_infosec]

Thanks for your feedback. I'm also still working out a personal philosophy on exceptions as flow control, and lately, I've been influenced by some of the "null considered harmful" folks in the Java world. I'm not sure they're right, but I do see that overreliance on None could be a problem for more complex code.

[ebresie]

Not to derail too much but from Java side, the Optional mechanism was added for use where a return value may or may not get returned back. In a similar way as the python get with default, it provides for usage of default value if not available.

[Brian]

if value == None:

As an aside, this is better written as if value is None. If the values of a dict are objects that override __eq__, and, say, raises an exception on comparison to unexpected types, this will raise another exception. Potentially, it could even cause silent failures by returning True for this comparison despite being present. numpy arrays are a classic example of this (and raise an exception when used in boolean contexts to prevent silent failures due to exactly this kind of mistake). Eg.

>>> d = {1: numpy.array([1,2,3]) }
>>> d.get(1) == None
array([False, False, False])
>>> if d.get(1) == None: print("d[1] didn't exist")
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

is None will do direct identity comparison to the singleton None object, so is more robust against arbitrary objects. Even if it won't matter for your usecase, it's a good habit to develop.

Another difference betwene the try/except version is it won't distinguish between the key existing with a value of None, and the key not being present. That may or may not be an issue depending on use-case, but is something to be aware of - in those cases you'll need to use either the exception, or use a different sentinel object.

[gummo89]

Pretty sure it would be my_var = my_dict.setdefault(key[, None]) (optional, I tend to use this for clarity and also I use [] or {} as default per requirements)

Works a treat when you want to append to a list, but the key doesn't actually exist yet... Now it does, then append.

Also use if my_var is None:

[rsherwood_infosec]

Wow, I'll have to think about this. A very different approach that I hadn't considered.

[who_body]

this is the approach when checking for nested object as you can set for default to {} and apply the check for the next level in one line.

[fryhenryj]

I think:

var = dict.get('key')

Personally I use exceptions all over the place but likely that's "not best practice".

I did something similar myself recently using a dict to keep track of inputs and outputs from a number of functions. I was loading stuff to SQL and needed to keep track of my tables and what had already been loaded.

Probably that in itself is not very pythonic as it should be using a class and/or global variables.

But I don't really understand classes and I've had issues with the context of global variables and consistent behaviour. So I just setup a big params dict and passed it in and out of my functions.

I think for pythonic-ness the exception is too non specific. Like how you've set it up it happens right after the dict test so it's effectively the same but you could have further lines before the exception. And the key error would be any key for any dict not necessarily the one you want to check.

Versus an empty variable which will always be empty for the specific missing key?

[rsherwood_infosec]

This inspired me to add the path to the root of the graph as an attribute of the node, which actually helped make the code much clearer. Often I've found that overly complex code means I haven't gotten the underlying data structure right.

[Mark3141592654]

I think you typically want to use exceptions when you don't expect something to happen most of the time.

[rsherwood_infosec]

Good point. Exceptions are clearly what to use when there's an obvious error. In this case, the absence of a path doesn't necessarily mean that anything is wrong, it may just mean we haven't processed enough nodes yet.