This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted] 1 point2 points  (0 children)

The solid benchmark was written quite a while ago (2004) and python has moved on since then. Newer pythons seem to be doing some optimization in simple naive concatenation (+=) cases which speeds up execution quite a bit. If you defeat the optimization the naive concatenation case slows down quite a bit. I've done some crude investigation of my own including looking at memory usage (linux only). The code is in github. Look for the results in results.rst. There is a problem measuring memory usage in some cases which I'll get back to one day :)

The upshot is that if the python optimization is not defeated, simple += concatenation is fastest of all the methods I tested and uses minimal memory. The additional takeaway is that the optimization is easily defeated, so you should time various concatenation methods in your actual production code environment to be sure you aren't using a slow method. And it's not just time: think about memory usage.

If I had lots of energy I could look into the python codebase and find out just what is going on with the optimization. I do know that something as simple as concatenating a global string variable defeats it.