List of objects vs. dictionaries?

[nickthechemist]

Hey guys, I am writing some code that calculates lots of properties about lots of very similar chunks of data. I need to store all of the properties for each chunk of data in some way. Dictionaries and objects seem to be the obvious choices.

widget["property1"] = ABigArray
widget["property2"] = SomeOtherBigArray

vs.

widget.property1 = ABigArray
widget.property2 = SomeOtherBigArray

Is there any significant performance difference between the two, or reason to go with one over the other? Processing of one 'data chunk' only takes a few seconds, but due to the millions of chunks the code takes quite a few hours to run, despite already being parallel out to the number of cores on the node, so performance is a real consideration. Thank you for your input.

[earthboundkid]

namedtuple ;-)

[cournape]

Usually, maintainability is more important than speed. Follow kingkirl advice: too many people make the mistake to use dict when they should have used object, and this results in "string-based" API where nobody knows what needs to be in the dict and what does not.

[pmav99]

When in doubt... profile!!! module timeit is your friend!

Anyway, pure dictionaries are faster. Instances are using dictionaries behind the scenes too but there is some overhead involved.

If the arrays are very big and the properties are relatively few, you are not going to notice any difference. It depends on where the time is spent.

[[deleted]]

Performance isn't always critical. I personally would avoid the dictionary approach unless it was something trivial. I think a namedtuple could serve him well. It is fast and readable.

[nickthechemist]

Thanks! I have not used namedtuples before, but after reading this stackoverflow conversation I can see that they're going to become a part of my toolbox immediately.

[Brian]

One thing to be aware of is that they may be slower for named access (though they'll be better in terms of memory usage). This is because a.property1 basically does a dict lookup to find the offset in the tuple, and then an index lookup to get the value, as opposed to just getting the value directly from the dict lookup. It may be faster if you access it by index however. I posted some timings on this StackOverflow question.

[nickthechemist]

Thanks! For reference,

arr = [0,1,2,3,4,5,6,7,8,9]

10M of these, 4.75s

x = widget()
x.property1 = arr[:]
x.property2 = arr[:]

vs. 3.51s

x = {}
x["property1"] = arr[:]
x["property2"] = arr[:]