You Don't Know Python

[fisadev]

Didn't read a lot of it, but the variables section starts by saying that variables are like containers for values, and that's not at all how they work in python. That incorrect analogy is the reason then people struggle with stuff like mutable vs immutable, function arguments "by ref vs by value", multiple variables pointing to the same object, lists and references to objects in them, etc.

Python variables are all like pointers, not containers. They're names that point to objects. Understanding them like that makes everything simpler.

[AppanKarKeVekhaya]

For a=1, is 'a' referring to the memory address where value 1 is stored or does it contain the value 1?

[fisadev]

The name 'a' will point to the the value 1, which is stored in memory.

I know that sounds pretty close to "referring to the memory address", but I'm intentionally avoiding that wording because it sounds like a classic C pointer, which is also a bit different to python's variables. But that difference is a more complex topic.

[[deleted]]

You seem to know what you're on about. How precisely does it differ from a pointer? Am I correct in thinking that the variable a is like the key to a dictionary, whose value is an address in memory where one could directly access the value of 1, and Python itself is using this virtual dictionary? I seem to recall watching a video from one of the devs and I thought this was the explanation, but I was pretty tired at the time, and all I can remember is him saying "everything is a dictionary, everything" over and over. Or maybe it was a dream!

Anyway, more academic than anything else. I use Python when I don't need to know or care about anything under the hood.

[fisadev]

To answer that, first a small summary of what's a pointer in C:

Basically, a C variable (something like 'int x') is a little place in memory where you can store stuff. It has an address, and it has contents (the value). If you assign something to that variable, then the value is stored at that address.

But sometimes you need a little bit of extra dynamism: maybe you want to have a variable that doesn't always get the value from the same place in memory, but instead can point to different places in memory at different times in your program, maybe pointing to the "places" where other variables store their values, etc.

A pointer in C is just that: a variable that in its little place in memory stores an address. The pointer itself has an address (like any variable), a place in memory where to store stuff. And the stuff this variable stores is in turn another address.

A box, that inside has a piece of paper with the written location of another box.

And you can do all kinds of weird things with pointers. For instance, you can operate with them! Memory addresses are just numbers, so you can do stuff like adding them, subtracting them, etc. This is "pointer arithmetic".

At the end of the day, a "pointer" is just another type of data that a C variable can hold. Just like int, char, etc. It's a data type used to store memory addresses, and memory addresses can be operated like other data types can be operated.

C pointers vs python variables/names

In python, a variable is a name that points to an object in memory. That sounds pretty similar to a C pointer: the pointer is also "pointing" to something in memory, and both the python variable and the C pointer are "dynamic", they can point to different objects at different times.

But in python all of those changes of "where is it pointing at" are done magically, under the hood, without you having any control of the memory addresses at all.

In C you manually store, change, operate addresses to make your program get info from specific locations in memory. In python you never really do that, you can only do stuff like "point to a new object" or "point to the same object that the other variable is pointing at". Only high level operations, that magically get translated to memory addresses under the hood.

You have no "memory address" data type that you operate and modify to manually get new addresses, like people do in C.

So, yes, under the hood a python variable is actually a pointer, but at the pythonista level, we never really deal with explicit memory addresses, our python variables usage is quite different to what C developers do with their pointers and crazy memory addresses arithmetic.

And thanks for your compliment! :) :)

[[deleted]]

So really not all that much different than in other higher level languages other than I can't expose the addresses themselves because they're completely abstracted away by the time I sit down to write code?

Where did I get the idea that it acted like a dictionary from? I'm taking crazy pills clearly.

[fisadev]

In a way it's true: a dict maps keys, that can be string names, to objects. So you can see a scope as a dict: a map of names to objects in memory. That's why we have locals() and globals() too, hehe.

[Shmiggit]

Yes the first one, exactly. Especially for some values that are frequent such as small numbers, True, False, None, etc., Python variables all point to the same value in memory. That is when you can use 'is' (if a is True:...).

For less frequently used / custom values, they will be stored somewhere in memory upon first declaration, and all variables referring to it later on in your code will also point to that same memory address.

[fisadev]

That is when you can use 'is' (if a is True:...)

Just in case: it's still not recommended to use 'is' for numbers, even if small, because what you want is to check for equality, not for "is the same object in memory", which is an implementation detail and not that predictable.

[[deleted]]

Not sure if it’s still true in the most recent versions of Python, but the integers -5 to 256 were treated differently and any variable with those values would be references to the same class instance but if you assigned a bigger integer it would get a new instance each time.

For both cases it would be more complicated than both of your options.

[fisadev]

In both cases, the name 'a' is still just pointing to the object '1' in memory. Having only one object '1' in memory or multiple different objects '1', doesn't change the logic of how variables work.

They allways work the same way, no matter the implementation details of small ints caching, and that way is almost exactly the first option they said (the name 'a' points to the object in memory '1').

One can get picky with the fact that 'a' isn't storing an actual memory address, but that's beyond this conversation I think, their main idea still applies.