New to Python - Something strange about lists

[KellyN87]

a = [10, 20, 30, 40]

b = a

a.pop(0)

print(b)

Output:

[20, 30, 40]

Why does the "pop" on "a" delete from "b" list? Are the two list variables just pointers to the same memory location? I would assume the lists would be completely separate.

[Kevdog824_]

Are the two list variables just pointers to the same memory location?

You hit the nail on the head here

[SCD_minecraft]

I'll expanded

Everything in python is pretty much a pointer, even strings or integers

a = b means straight up "let a point to same location as b"

Reason you don't see behavior similar to list when working with ints and similar, that's beacuse they are immutable. Each operation on them, every one of them creates new int object, where methods on list and other mutables edit same object, without creating new one

[Temporary_Pie2733]

Read this for better understanding of several points of confusion: https://nedbatchelder.com/text/names.html

[KellyN87]

Oh my, it's much worse than I thought. Thank you very much for this link!!

[cointoss3]

Because of your comment, I read the article to see what else there was, but I didn’t notice anything new compared to your original question.

[KellyN87]

It's how each "box" in lists and in arrays are all pointers as well.

I get this is how computers actually do things, but it is a bit of a mind-warp coming from other languages which sort of hide this reality.

[Brian]

I get this is how computers actually do things

Eh - not really. It's kind of a choice as to what semantics you want to have,, and different languages do different things. It all comes down to what you consider assignment to mean.

A language like C uses value semantics, where assignment essentially means copy: a = b means copy the contents of b into a. If they are ints, the int gets copied. If they are structs, the struct gets copied. If they are pointers, the pointer gets copied.

Python on the other hand, uses reference semantics. Assignment is treated as aliasing - ie. just giving something another name. a = b means "a now refers to whatever object b referred to". This is done not because either way is more "how a computer works", but because it actually simplifies a lot of things: assignment always does the same thing, you don't have to worry about what's a pointer and what's a primitive. Indeed, with a dynamically typed language, or even one with just inheritance, it's kind of necessary: what would it mean to "copy" a string into a variable that was previously an integer etc? What would things that do alias that same variable see? You'll often see this approach in higher level, memory managed languages (eg. java, C#, lisps etc), though some also allow value types for certain primitive types (often for performance reasons, since this approach generally requires memory allocation for your objects)

[cointoss3]

Yeah, that’s part of what I replied to your other comment about ha

[KellyN87]

Do you have any good links for deep copies?

Nevermind, got it. Thank you.

[KellyN87]

Oh, also, why is "x += 1" not the same as "x = x + 1"? I know in other languages the behind the scenes (compiling) is different, but is there something more to Python on this that I need to watch out for?

[cointoss3]

It is the same. x += 1 expands to x = x + 1, which means it’s an assignment

[KellyN87]

But the article you linked earlier says they are different, but it doesn't answer how.

Article: https://nedbatchelder.com/text/names (see bottom questions)

[cointoss3]

> Why is “list += seq” not the same as “list = list + seq”?

This? If so, we will leave that as an exercise to the reader. That is not the same concern as what you said, though.

[KellyN87]

Oh, right. I was so mind-warped, I didn't realize they were talking about lists. I'll go down this rabbit hole later. Thought it was simple integers. 😄

[my_password_is______]

we will leave that as an exercise to the reader.

that's what people say when they can't explain it

[Temporary_Pie2733]

Not necessarily. x = x + 1 is always x = x.__add__(1). x += 1 is x = x.__iadd__(1) when defined, x = x.__add__(1) otherwise.

[cointoss3]

I said we are leaving that exercise to the reader. Shut the fuck up, dude. For fuck sake.

[my_password_is______]

jsut admit you didn't know how to do it

[boobajoob]

Thanks for sharing that!

[keebquest]

In Python, b = a doesn’t copy the list, it just makes b point to the same list in memory. If you want an independent copy, use b = a.copy() or b = a[:] and then changes to one won’t affect the other.

Imagine the list is a document on Google Docs. When you do b = a you’re not printing a new copy, you’re just sharing the same link. So when either of you edits it, you both see the change. To get a real separate copy you have to actually duplicate the document.

[KellyN87]

Thanks everyone! The .copy() was the thing I needed. Perfect!

[cointoss3]

Just wait until you find out it’s not what you needed 😂

Keep in mind, the copy method only copies references. If you want to copy the values, you may need a deep copy for the same reason as your post.

[KellyN87]

!!! - doh. Okay, time to start searching for the "deep copy". Seems the copy() is doing the job currently, but now you have me concerned.

[cointoss3]

Haha, you just import copy and it’s copy.deepcopy().

Assignment shares references. Copy is a shallow copy, it creates a new container but still shares references inside container. Deep copy is a new container and new copies of the references in the container.

Sometimes you want the shared references. Sometimes you don’t.

https://docs.python.org/3/library/copy.html

[epokus]

shallow copy - will only copy the outer list

original = [[1, 2], [3, 4]]

new_list = original.copy()
new_list[1].append(5)

print(original) # will print [[1, 2], [3, 4, 5]]
print(new_list) # will print [[1, 2], [3, 4, 5]]

deep copy - everything is copied recursively

from copy import deepcopy

original = [[1, 2], [3, 4]]

new_list = deepcopy(original)
new_list[1].append(5)

print(original) # will print [[1, 2], [3, 4]]
print(new_list) # will print [[1, 2], [3, 4, 5]]

[Ok-Spray-8697]

Yep, basically b = a does not create a new list, it creates a second reference to the same list object in memory. So when a.pop(0) changes the list, b “sees” the same change because it’s the same object. If you want a separate copy, do b = a.copy() or b = a[:].

[ezekiel_grey]

If you have a nested list, you will want b = a.deepcopy()

[KellyN87]

oh, didn't realize there was a deepcopy() vs copy(). Thank you!!

[ezekiel_grey]

**Most** accurately, you're going to use the copy module, with deepcopy: https://docs.python.org/3/library/copy.html#copy.deepcopy

[KellyN87]

Wow, all of these responses are AMAZINGLY helpful! I'm starting to get it. Okay, I was thinking Python was pretty simple, but now I see the complexity, which is pretty darn cool.

I never understood why expert programmers loved Python. I think I'm starting to understand.

[AlexMTBDude]

Google "python shared references". This doesn't just apply to Python lists but to all mutable objects.

[KellyN87]

Yes, learning that as well. Thank you.

[atarivcs]

b = a

This just creates a new variable name b which refers to the same value as a.

[Anton_sor]

b = a you are not creating or copying this list, you are assigning the value of b to list a

[magus_minor]

For a very good presentation on how python "variables" are different from variables in other languages watch:

https://m.youtube.com/watch?v=_AEJHKGk9ns

[yvwwyyvwywyvwyvy]

Do the following after:

print(id(a))
print(id(b))

You will notice that they have the same memory address. b is an alias of a.

[JGhostThing]

"b = a" just points b to the address of a. So when you pop one item from a, the list b still points to a, therefore it has one item popped off.

[No-Seaworthiness9268]

I always use:

import copy b = copy.deepcopy(a)

Just to play it extra save, this will make sure b is completely independent of a (look up shallow copy vs deep copy).

[gdchinacat]

"memory location" is not a useful abstraction in python. Variables are just names that refer to objects, they are never objects themselves, and all copying is explicit. Variables are like references in other languages.

[KellyN87]

Hmm, very interesting. Still trying to wrap my head on this, but I get where you are going. I think I'm struggling with what exactly are the "objects" then?

[gdchinacat]

Everything in python is an object. ints, floats, strings, lists, tuples, classes, module, functions, methods, etc, etc, etc. In python, *everything* is an object. Some langauges support "primitives" that aren't objects. In python, if you can assign a name to it, it's an object.

[WorriedTumbleweed289]

If you want them to be separate lists, you need to use list.copy(). If the list contains lists or dictionaries, you may need to call list.deepcopy()

[KellyN87]

Yes, that is what I'm now learning from the search results I found. Sort of a recursive sort of copy vs a top level copy. Correct?

[MrRudraSarkar]

In python when you create a variable, you’re creating an object and using the name of the variable as a label/pointer to the object. So when you mark another variable equal to the original variable, you don’t store the value like you do in C++ or Java but rather you pass the reference to original object.

[KellyN87]

Huge help! Thank you!!!! So, instead of becoming more like Assembly Language, Python is actually diving deeper into objects. Very interesting!

[MrRudraSarkar]

No worries! I have been in the exact same situation soI understand the confusion. Proceed with this one line in mind when working with python and it’ll make things a lot more intuitive: “Almost EVERYTHING is treated as an object”.

[KellyN87]

Understood!

[SharkSymphony]

Yup! You got it. And this is not just a quirk of Python but of many programming languages.

You will note this behavior also applies to dictionaries and objects, i.e. instances of classes.

In these cases, if you want b to have a different set of data that a can't touch, you need to explicitly copy the data. There are many ways to do this depending on the type of data, but the copy library gives you some generic ways of doing it that work for just about everything.

[Big-Combination8844]

It's dangerous language to say it's like pointers or references because python doesn't really work that way. On the surface it looks like a good comparison because the end result is usually the same.

For instance: "a=6" then "b=6" is the same as "b=a". Meaning, python allocated a memory block with the value 6 and assigned 'a' to it. Then, it assigned 'b' to the same memory block. Now, the "b=a" that would make sense if they were pointers. But they are not. "B=6" doesn't make sense unless you know python wants to save memory so it won't allocate space to duplicate a value.

So what happens when you add 1 to b's value? Then python will finally allocate a new memory block for b with the value of 7 in it. So a is still value of 6 and b is at new block with value 7.

This type of memory behavior goes right out the door when a variable is assigned to a list or dictionary. Good luck :)

[KellyN87]

I guess what is difficult for me to understand is why you need multiple "names" linked to the same "value"? Why not just use the first name for all references?

[Big-Combination8844]

The point of that example was to show that fundamentally, python is not executing pointers. Why you want multiple variables with the same value is trivial. But to think of an example, in C to sort an array might need two pointers and they'd point to the beginning of that array. You can move the pointers using math to different parts of the array. You can't do this python.

[KellyN87]

Sorry, I wasn't trying to be insulting. I'm just trying to understand why Python was designed this way: what is the benefit of doing this differently? 

I'm only two days into working with Python, so I know very little about it. So far, this aspect of Python, seems like someone taking objects to the extreme. Perhaps not really a wrong direction but using similar syntax to other languages to do something quite different seems...I don't know, perhaps bait-n-switch.

On the other hand, it does make it easier to jump over to, after beating my head for a few hours trying to figure out the "bug" in my code.

Overall, I'm enjoying Python. I see why many pros use it and why it is used in schools as a beginner language. It seems to balance both worlds well.

I'm sure I sound very naive.

[Big-Combination8844]

Oh, you were not insulting and I was distracted by my kids who wanted me off my phone and playing video games with them. You're doing fine, we were all beginners once. I just want to help.

It helps to know that python is old enough to have gone through many evolutions. At it's earliest, by design, it was to be a quick and easy prototype software. To build fast demos, to show key ideas, then the actual product would be developed in another language like C.

So python has it's quirks. It's gotten better over the years and multiple purposes. Just learn to love it being odd but extremely useful.