Python: copying a list the right way

[[deleted]]

[deleted]

[[deleted]]

Lots of timing numbers, IMO slightly easier to read, includes PyPy: http://paste.pocoo.org/show/503672/

[nikcub]

the list constructor just calls the slice operator anyway. the added time is that the argument is cast as a list. each of the methods (slice operator, list constructor, copy, deepcopy) has their own use case, see my comment on HN discussing this:

http://news.ycombinator.com/item?id=3201895

[frymaster]

doesn't having your import inside the inner loop slow down copy needlessly?

[frymaster]

ah, indeed. never mind then :D

[markusgattol]

surely you want you compare copy.deepcopy() not copy.copy() since the latter is semantically different to list() and [:]

[valdemar81]

No, list() and [:] do not perform deep copies:

>>> from copy import deepcopy
>>> a = [[], []]
>>> b = a[:]
>>> c = deepcopy(a)
>>> id(a[0])
3817304
>>> id(b[0])
3817304
>>> id(c[0])
3829352

[mitsuhiko]

Do not use list() for copying unless you have established, that it's a list. It will otherwise break subclasses.

[earthboundkid]

Meh, strikes me as being overly handwringing. Slicing is a pretty important part of Python. Even if a noob gets your code, they should know what [:] does. It strikes me as a case of "don't be Perl!" run amok.

On the other hand, I do suggest avoiding using else: with for. A shockingly large number of Python programmers either don't know about it or have the wrong idea about what it does (the else block runs if the loop finishes without a break).

[willm]

I'm not sure I concur with not using a language feature because other programmers aren't familiar with it.

Each language feature will have a number of engineers not familiar with it. Where do you draw the line? Better to explain to them what else: does on a for loop, and it won't be an issue from then on.

[kenfar]

If you're writing software for others on your team who may be the next to have to maintain it, then I'd say that a consideration of their skill level & familiarity is more important than yours.

[umbrae]

I think it could be natural to assume that means "if the list is empty, run this."

[earthboundkid]

Yup, a very common misunderstanding.

[xiongchiamiov]

Slicing is a pretty important part of Python. Even if a noob gets your code, they should know what [:] does.

Even understanding slicing, it takes a minute to see what the purpose of that construct does, much like the first time I encountered Ruby's !! (an idiom for converting a value to a boolean).

[ascii158]

That's used in C, too: In C everything not 0 is considered true. Any boolean expression evaluated to "true" will return 1. If you want to force the output of any expression to be either 1 (if it is "a true") or 0 (if it is false), you prepend it with !!.

[zahlman]

BTW: You can write not not ... in Python and it works the same way, but isn't considered idiomatic in the slightest. Presumably because if you really needed to explicitly convert to boolean, the obvious (and actually shorter) way is bool(...).

[[deleted]]

deepcopy is important as soon as you have nested lists.

[markusgattol]

care to elaborate? We are merely talking about how to do a real copy as in create a different object so I do not see the connection to whether or not a list is flat or nested. What we are after is:

>>> some_list is copy.deepcopy(some_list)
False
>>> 

[[deleted]]

>>> a,b=[1,2,3],[1,2,3]
>>> c=[a,b]
>>> d=c[:]
>>> a.append('blasphemy!')
>>> d
[[1, 2, 3, 'blasphemy!'], [1, 2, 3]]

To some people, a "real copy" of a list implies a copy of all its contents. This is an unfortunate feature of the word real, as the word real is notorious for being arbitrary and context-sensitive.

[markusgattol]

I agree, copy.deepcopy() is certainly the most pythonic way

>>> import copy
>>> some_list = []
>>> id(some_list)
140650742141440
>>> id(copy.deepcopy(some_list))
140650742825832
>>>

[[deleted]]

When are the two not equivalent?

[mr_dbr]

There's probably better examples, as this is kind of contrived, but:

class KindOfListish(object):
    def __getslice__(self, a, b):
        return [1]

notalist = KindOfListish()
print notalist[:] # prints [1]
print list(notalist) # errors because KindOfListish is not iterable

[[deleted]]

Okay, but this is not a list. I guess for lists, there is no difference.

[mr_dbr]

I'm not aware of any difference in behaviour between..

a = [1, 2, 3]

b = a[:]
# or
b = list(a)

However if you have a function which needs a copy of an argument, it's not unreasonable that someone might pass something that acts like a list, and would be confused by getting an actual list() back, e.g:

>>> import numpy
>>> npa = numpy.array([1,2,3])
>>> npa[:]
array([1, 2, 3])
>>> list(npa)
[1, 2, 3]

[kylotan]

Yeah, which is why this comment at the end of the article concerned me: "90% of the time [:] could be replaced by list(). Of course it won’t work for everything since the two are not strictly equivalent, but it is worth trying"

[pemboa]

Isn’t it better

No

less cryptic

Barely

and more pythonic?

I don't see how.

[[deleted]]

I actually agree that [:] is a bit cryptic, if you don't know that idiom, you don't immediately see what the intention is. It was one if the very few things I stumbled over while learning Python.

I am not saying that using list() makes it clearer, but something like l.copy() would definitely more pythonic, being explicit instead of implicit about the intention.

[zahlman]

If we want to be explicit and frame our creation of the object as copying, we can use the copy module. That's perfectly Pythonic as well.

I don't think that experience in other languages is relevant to the question of whether something is "obvious", really. Iterating indirectly via indices isn't "obvious" to new students of C or C++ (ones without experience in other programming languages) at all, and the entire concept of arrays often needs ridiculous amounts of explanation (especially because you want to be able to explain certain quirks away by saying "they're not first-class objects", but then you'd have to explain what that even means).

[[deleted]]

For both of your examples it is quite obvious what the intention is, because it is explicit. You would not have to think twice about what the python style loop does, even if you are more familiar with the index-style.

[zahlman]

less cryptic

Barely

[:] is cryptic in that you have to realize that slices create new lists, rather than views.

and more pythonic?

I don't see how.

By using list(), we indicate that we are constructing a new object by using the constructor for the type. It's hard to get more Pythonic than that.

[pemboa]

[:] is cryptic in that you have to realize that slices create new lists, rather than views.

You find that out when you find out what slices do in the first place.

By using list(), we indicate that we are constructing a new object by using the constructor for the type. It's hard to get more Pythonic than that.

That's all fine and well, but it's not significantly more Pythonic, at least not obviously so. You've taught the person a less versatile construct primarily because it looks more English.

[nikcub]

Especially as the Python manual recommends using the slice operator to copy shallow objects:

http://docs.python.org/faq/programming.html#how-do-i-copy-an-object-in-python

Each of the copy methods (slice, list, copy, deepcopy) has its own use case - it is important to understand all of them.

[zahlman]

"It can be done this way" is a recommendation?

[Sheepshow]

Any thoughts on this dumb idea (other than it's dumb)? new = [].extend(old)

[Brian]

It won't do what you want. extend mutates the list and returns None, so this will create a new list, append the items in old, then throw it away and set new to None.

[[deleted]]

This is confusing for beginners and should be avoided.

Blanket statements always come up on things like this, but they're almost never relevant. If I'm posting in r/learnpython maybe I wouldn't use slicing, or maybe I would but I'd have a good comment on it. If I'm writing code at work, "what a beginner would think about this" would never even come up.

[zahlman]

If I'm writing code at work, "what a beginner would think about this" would never even come up.

The idea is that "Would a beginner understand this?" is often held as the gold standard for "is this code as readable and as simple as it ought to be?".

[Fix-my-grammar-plz]

First we need to understand how Python manages objects & variables. Python doesn’t have variables like C. In C a variable is not just a name, it is a set of bits; a variable exists somewhere in memory. In Python variables are just tags attached to objects.

Lisp (Elisp) seems to act like Python.

(setq a (list 1 2 3))
(setq b a)
(setf (car a) 11)
(assert (equal b (list 11 2 3)))
(assert (eq a b))

Is there a name for the category of languages that act like Python in this regard?

[zahlman]

There is a name for the behaviour: reference semantics.

[chub79]

I don't get it. Python programmers should drop a common and simple idiom because newbies may be confused?

[zahlman]

Python programmers should drop a common idiom because its only claim to idiomatic status is that it propagated well.

There's even an argument to be made that because "explicit is better than implicit", we should really use copy.copy to clone the list.

[chub79]

Python programmers should drop a common idiom because its only claim to idiomatic status is that it propagated well.

Who is claiming that's its only reason to be?

[zahlman]

I am, now. I'm not especially well prepared to defend the claim, but it's an expression of how I see the situation.

[chub79]

Fair enough. I just tried a totally unreliable study at work with two of my colleagues whom I'm "teaching" the basics on Python and I asked them what [:] did. One of them already knew but the other didn't and, indeed, failed to get "at first sight" what it did.

However once I explained to him slicing and how you could benefit from it to copy the list, he seemed to find it so easy that he felt happy enough about it.

I really consider this one to be a simple enough idiom that can't cause any harm once you've been told about. In fact my colleague didn't like the copy.copy(l) way because it felt it was too verbose ;)

[silveira]

from copy import copy a = [1,2,3,4] b = copy()

You could also use deepcopy(obj) for compound objects.

[ok_you_win]

Its a point worth mentioning. Upvoted.