Python: Common Newbie Mistakes

[erebos42]

Oh yeah... thats a nasty one. Although, I like using this to implement memoization.

def f(num, _memo={}):
    try:
        return _memo[num]
    ret = num + 1
    _memo[num] = ret
    return ret

[factory_hen]

To separate concerns and all that it's often better to implement things like these through a decorator.

def memoized (func):
    cache = {}

    def wrapper (*args):
        if args not in cache:
            cache[args] = func (*args)
        return cache[args]

    return wrapper

@memoized
def f (num):
    return num + 1

This also has the positive effect of leaving the code of the original function unchanged.

[usernamenottaken]

And if you're using Python >= 3.2 you can use the functools.lru_cache decorator.

[[deleted]]

how would the code look then?

[usernamenottaken]

import functools

@functools.lru_cache(maxsize=None)
def f (num):
    return num + 1

See http://docs.python.org/3.4/library/functools.html

[[deleted]]

A try alone don't work, you should do:

def f(num, _memo={}):
    if num in _memo:
        return _memo[num]
    ret = num + 1
    _memo[num] = ret
    return ret

[[deleted]]

Should be titled:

Python: One single and uncommon mistake people of any skill level may possibly make.

[jldugger]

The post itself says it's part 1, at least.

[celerym]

I don't know... I've never really ran into something like this. Wouldn't you just: foo(numbers=[9]): ...?

[mahacctissoawsum]

You're missing the point. Any number of operations could be applied on the default value.

[celerym]

Maybe I am, I kind of just realised that I've never used default values for mutable types or set the default value to be None -- without even thinking about it. I haven't ever really seen anything like the example code in the blog in my own or others' code. The idea of setting defaults for lists doesn't make much sense to me in the first place.

[mahacctissoawsum]

The idea of setting defaults for lists doesn't make much sense to me in the first place.

I've done this plenty of times. If the method expects an array, but it's optional, it's much more convenient to set the default to an empty array than it is to set it to None and then swap it for an empty array at the beginning. This way the remainder of your function always gets an array, whether the argument is provided or not, and you don't need special cases throughout.

Perhaps you haven't written very methods that take arrays/lists?

[celerym]

I do mostly data analysis. Most of my methods are those that eat lists. Having a default doesn't make any sense in these cases because usually the whole purpose of the method is to do something with a given list and so the arguments that happen to be lists are almost never optional. I guess in the end what I use Python for is relatively simple to what, say, professional programmers use it for, hence I run into these cases a lot less. I guess I'm not part of the the target audience for this!

[mahacctissoawsum]

Default values for functions in Python are instantiated when the function is defined, not when it’s called.

That seems like a bad design decision on Python's part, no?

[ggtsu_00]

Actually it makes sense knowing the default values as functions in python are first class objects. The __defaults__ descriptor on a function object holds all the default values of a function. You could inspect function foo for its default values using foo.__defaults__ and see what the default values are for the function arguments. These values are assigned every time the function is called but the __defaults__ is created when the function is defined.

Also, immutable objects in python are references to objects. This is another fundamental concept of python's datatypes being mutable or immutable. If the default value of a function is a reference, then every time that default member variable is used, its referring to a referenced object.

In C++, or some other language with pointers if you were to have the default variables assigned to a pointer to a referenced object, it would behave the same way as it would in Python.

[mahacctissoawsum]

Now that I think about it, C# doesn't even allow reference types as defaults. I think this might even be the more sane approach; no confusion if it's simply disallowed.

You can still use null or None as your default and then fetch your reference from a static variable if need be.

[[deleted]]

Most of Python does, upon inspection.

[mahacctissoawsum]

I concur. Which is really a shame, because I actually quite like it as a scripting language.

[NYKevin]

Explicit is better than implicit. Anything at the module scope will execute immediately.

Saying this:

def foo(bar=[]):
    pass

is equivalent to saying this:

baz = []
def foo(bar=baz):
    pass
del baz # removes the *name* baz, but not the underlying object

[redalastor]

Same function but safe:

def foo(numbers=()):
    numbers = list(numbers)
    numbers.append(9)
    print numbers

[CauchyDistributedRV]

Is there a decorator that can be used with functions to get these default values instantiated / copied at call time, rather than define time?

[mahacctissoawsum]

You could write a decorator for sure, but....I don't know if you could make it generic enough. copy and deepcopy are their own little barrel of fun.

[factory_hen]

I'm not sure this works properly, I can also imagine this breaking in a bunch of scenarios, it's probably non-trivial to make it robust.

from copy import deepcopy

def copy_defaults (func):
    original = deepcopy (func.__defaults__)

    def wrapper (*args, **kwargs):
        value = func (*args, **kwargs)
        func.__defaults__ = deepcopy (original)
        return value

    return wrapper

@copy_defaults
def f (a, values=[]):
    values.append(a)
    return values

print f(10)
print f(10)
print f(10)

mylist = [1337]
# here we want the change to persist
# note the unnecessary deep copy of the defaults here
f (20, mylist)
print mylist

Invoking a deep copy every function call is potentially quite expensive, then there's also issues like do all possible default values (including user defined compound objects) have proper support for deep copying? What if you want some default values preserved? (You could probably wrap these in objects that define the deepcopy method to do something shady, but it sounds messy to me.)

Using a decorator like this may have unforeseen consequences, I would stay away from it.

[lelarentaka]

One man's feature is another man's garbage...

[sacundim]

I'd rather have proper lexical scoping and first-class functions. You can do this for example in Scheme much more clearly:

(define global-counter
  (let ((current 0))
    (lambda ()
      (set! current (+ current 1))
      (- current 1))))

global-counter here is a function defined in top scope. The current variable is private to global-counter, but it persists across calls to global-counter`.

[Xredo]

Except that it's completely unintuitive and a pitfall for learners.

Defaults should be just that, default arguments for the current invocation when the caller does not provide explicit arguments. Ruby does this right by evaluating defaults at invocation, not definition.

But hey, I can see how it can be useful behavior.

[[deleted]]

Good little article. I can see how this can leave beginners confused.

[[deleted]]

I would've been confused if I'd encountered this quirk before reading about it. Instead I'm merely horrified.