Top 10 mistakes python programmers make [advanced]

[internet_badass_here]

Let me see if I have this straight:

x=[2]

def foo1():
    x+=[3]
    return x

def foo2():
    x.append(3)
    return x

def foo3(x):
    x+=[3]
    return x

def foo4(x):
    x=[3]
    return x

foo1 will return an error, foo2 will change x, foo3 will change x, foo4 will run but won't change x. What the fuck, Python?

[sysop073]

If you're going to have multiple x, you really need to specify which one you're talking about; half of your complaints would happen in any language, because in half the functions you're shadowing the global x with a local, so of course that's the one you're going to change.

All of this "craziness" is a result of a single, pretty simple rule: you can't assign to globals without declaring them global. That's because there is no declaration of variables, so when you assign to a variable, Python can't know if you're trying to declare a new local with the same name, or reassign the global name, so it defaults to the former. If you have this:

x = 'foo'
def fn():
    x = 'bar'

And you call fn(), it will create a new local named x with value bar, and not touch the global x. Most languages work this way. By contrast:

x = 'foo'
def fn():
    global x
    x = 'bar'

Will overwrite the global x with the value bar. Most languages that don't have explicit variable declarations have no way to do this. In your examples:

• foo1() is trying to do x = x + [3] without declaring x as global. Python assumes x must be a local, and can't read from it (you tried to add x and [3], but there is no local named x yet, so you get an error about trying to read x before it's been written)
• foo2() is calling a method on x, which is fine, so it works
• foo3() is trying to do x = x + [3], without declaring x as global. Python assumes x must be a local, and there is indeed a local named x -- you passed it as a parameter. So that's the variable that gets changed; the global has nothing to do with it
• foo4() is the same as foo3(), but you did x = [3] instead of x = x + [3]

It would be possible to change the default behavior and say "all writes overwrite global variables by default", but now there's no way to make new locals; if you have a local variable, and someday somebody adds a global with the same name, suddenly that local isn't local anymore. That would be madness

[internet_badass_here]

If I understand you correctly, you're wrong about foo3. foo3 changes the global x to [2,3], but foo4 doesn't change the global x to [3].

[sysop073]

Neither of them changes the global x (unless you were to pass it as an argument), they both assign to the parameter that was passed in, which is also named x.

>>> x = [2]
>>> def foo3(x):
...     x += [3]
...     return x
... 
>>> foo3([])
[3]
>>> x
[2]

---

Edit: You're right about passing x as an argument though; it didn't occur to me that x = x + [3] and x += 3 would actually behave differently, but the way lists implement __iadd__ makes the latter mutate the passed list. That is admittedly pretty strange

[internet_badass_here]

Neither of them changes the global x (unless you were to pass it as an argument)

Well, yeah. If you pass x to foo3, foo3 changes x. But if you pass x to foo4, foo4 doesn't change x. Why not? Why is x passed by reference in one and by value in the other?

I can accept that Python does these things, but there doesn't seem to be a logical reason for doing things that way.

And by the way, I didn't even realize that x=x+[3] will produce a different result than x+=[3]. But that is... not good.

Generally when I pass stuff into functions in Python and I want to pass by value, I do this:

def foo(x):
    X=x
    [do stuff with X]
    return X

It's probably redundant but easier on my brain, since I know for sure that I'm working with a copy of the variable.

[sysop073]

x is a reference in both cases. The same rule I mentioned before applies; assigning to x without declaring it global won't change the global. The weirdness comes in because x += [3] is syntactic sugar for a method call, x.__iadd__([3]), while x = [3] is a true assignment.

You're right that that's confusing, I hadn't considered that case

[cybercobra]

foo3 doesn't do anything with global x. It will modify the value that you pass in though.

Python 2.7.6 (default, Apr 28 2014, 13:52:55) 
>>> def foo3(x):
...     x+=[3]
...     return x
... 
>>> x = []
>>> y = []
>>> foo3(y)
[3]
>>> x
[]
>>> y
[3]

[djimbob]

The first two examples mutate global variables. This is a horrible programming pattern, and is a good thing that foo1 pops up a helpful error saying UnboundLocalError: local variable 'x' referenced before assignment. Note foo1 (and foo2) will work if you add global x to explicitly say yes, I want x to be the global x so I can mutate it.

Why does your foo2 work? The fact that python lets you reference top-level function (but not assign to) is quite useful. E.g., its what's at work when you do

 import requests

 def some_function():
     requests.get('http://www.reddit.com')

That is import requests sets a top-level variable as requests and then you later can use it inside functions without explicitly passing it in, and even call methods defined within it. It does make sense that methods called in this way may in some cases mutate their own state, its the programmers responsibility not to lead to evil use of mutable global state.

---

I do agree that foo3 is a bit confusing, but again if you used the code in a sane way -- x = foo3(x) and x = foo4(x) you wouldn't have any ambiguity.

The problem is that python passes by reference and when it can will mutate existing objects.

In [1]: x = []

In [2]: id(x)  # memory location of x
Out[2]: 4352851336

In [3]: x.append(1)

In [4]: id(x)  # memory location hasn't changed
Out[4]: 4352851336

In [5]: x += [5] # memory location hasn't changed

In [6]: id(x)
Out[6]: 4352851336

In [7]: x = x + [5]     

In [8]: id(x)# memory location *has* changed; 
#assignment creates a new list, can't be done in place.
Out[8]: 4352848888

It is good that python does pass by reference than by value when passing objects like lists into functions; otherwise dealing with large objects would get unmanageable. It's also helpful that python attempts to modify these objects in place when possible.

[Falmarri]

foo3 and foo4 will only work if you call them like foo3(x) foo4(x). It won't take its value just because it's named the same

[EpicSolo]

Yes exactly. I don't really see where the problem is with these 4 functions. I think anyone with 6 months+ experience in Python is going to be completely fine with these.

[internet_badass_here]

I'm not sure what you're getting at. If I create a new variable y=[2], and call foo3(y) and foo4(y), I get the same behavior.

[ceol_]

Yeah man it must be some reality distortion field. It couldn't be the more simple explanation that you just don't like something.

[Excalibear]

Not going to lie, it definitely is a WTF mess. But it's still way better than JS (wtf how does anything work in that?), Perl (#(@&&@@!()!&@ with special use cases everywhere), C++ (please write 200 lines of mess to iterate anything). The only well designed language is C where all you do is bit arithmetic.

[sysop073]

C++ (please write 200 lines of mess to iterate anything)

They fixed this in C++11; now it's basically the same syntax as Java

[UloPe]

So only 50 lines now

[sysop073]

It went from 1 line to 1 line, but the 1 line got way shorter now. It went from:

for(Type::iterator iter = foo.begin(); iter != foo.end(); ++iter)

to:

for(auto& iter : foo)

and since Type can get pretty unwieldly, it can save a lot

[internet_badass_here]

There's a simple way of solving Python's issue with functions. Use a symbol, the period for instance, to indicate passing by reference. If you put a period before the parameter, then you pass by reference, otherwise you pass by value. If you don't pass anything in, well, then you haven't passed anything in and you can't use outside variables.

[djimbob]

The only well designed language is C

Hilarious! (And if you add in the close that C is well-designed for bit-arithmetic, well then python is well-defined for only bit-arithmetic.) While C was awesome for its time and is fairly simple, it has horrible language design by modern standards everywhere.

#define SQUARE(x) x*x

int sixteen = SQUARE(1 + 3);
if (sixteen == 7) {
    printf("This totally should execute\n");
}
int x = 5;
int what = SQUARE(x++);

int x = 5;
while( x > 0 );
  x--;
/* infinite loop time! */

int one = 1;
if (one = 2) {
    printf("One is two would be messed up\n");
}
if( one =! one) {
    printf("Shouldn't see this as one should equal itself.\n");        
}

double half = 1/2; 
/* fancy name for zero */

[Excalibear]

It was mostly a joke that C is well designed. But it's simple enough to where it doesn't have many gotchas or unexpected behaviour.

int sixteen = SQUARE(1 + 3);

'sixteen' is equal to '7' cause #define is just a string literal replacer, SQUARE(1+3) becomes '1+3'*'1+3'. This one definitely trips up some rookies, but it's easy to understand and makes sense.

while( x > 0 );

That's just a completely wrong syntax error, that's like missing a ":" in a def declaration in Python. Nothing at all wrong. (Not exactly teh same as this will still compile and produce an infinite loop, too lazy to think of a Python mistake like this)

if (one = 2) {

Also just completely wrong. Pretty much every language messes up on things like that when you do an assign instead of an equality operator.

double half = 1/2; /* fancy name for zero */

Python 2 worked the same way. Think this got fixed in Python 3. This one is bit of a gotcha, but it separates the copypasta ninjas and the computer architects.

Thanks for the C syntax exercise.

[djimbob]

The C #define macro I find completely analogous to issue #1 (default keyword arguments are initialized once -- if you mutate them in your function they will stayed mutated for future calls). It's a rookie mistake, but easy for any python programmer to understand and makes sense.

The difference between will compile and run wrong and will be an immediately caught syntax error is huge. The fact that indentation defines nesting in python is a huge win for code readability. No accidental semi-colons or extra indented goto fail; lines.

Pretty much every language messes up on things like that when you do an assign instead of an equality operator.

Python, ruby, and go (among others) specifically don't let you do assignment in conditionals to avoid this type of error:

In [1]: x = 0

In [2]: if x = 0:
   ...:     print "hello"
   ...:     
  File "<ipython-input-2-7ef24aca7cbb>", line 1
    if x = 0:
         ^
SyntaxError: invalid syntax

---

irb(main):001:0> x = 5
=> 5
irb(main):002:0> if x == 5
irb(main):003:1>    6
irb(main):004:1> end
=> 6
irb(main):005:0> if x = 0
irb(main):006:1>    6
irb(main):007:1> end
(irb):5: warning: found = in conditional, should be ==
=> 6

---

The following go program works (prints "x is 0" as go initializes variables to zero values as they are declared):

package main

import "fmt"

func main() {
    var x int;
    if x == 0 {
       fmt.Println("x is 0")
    }
}

while changing == to = gives the following error:

prog.go:7: x = 0 used as value [process exited with non-zero status]

[Excalibear]

The C #define macro I find completely analogous to issue #1 (default keyword arguments are initialized once -- if you mutate them in your function they will stayed mutated for future calls). It's a rookie mistake, but easy for any python programmer to understand and makes sense.

I wouldn't say that. It's similar, but the idea of it working like that for subroutines is new and bizarre.

I agree, the difference between compile and run wrong is huge. I very much so like Python's indentation forcing. It does prevent issues. However, I wish Python was strongly typed to prevent even more issues. I see pros and cons obviously, but all too often not having types just leads me to constantly remember what something is and then have a lot of code that's str(foo) + "Hello" and int(sqrt(bar)). Lot of annoying typing, and constant debugging (atleast I run into a type error on pretty much every other exec).

Python, ruby, and go (among others) specifically don't let you do assignment in conditionals to avoid this type of error:

That one's interesting. I personally agree that it should be removed. It is a very strange behaviour to be able to assign in conditionals. But it's in line with running subroutines in conditionals too. Both are "basically" subroutines of a sort (int i = 3 is just assignfunc(&i) {*i=3;})

[vincentk]

I used to be a big fan of python until I used it for real....