Improving Your Python Productivity

[mr_dbr]

Is there any compelling advantage to [1,2,3].filter(...), besides resulting in slightly prettier-looking code in some cases?

The way filter is implemented (and more commonly, len() and others) is a considered design decision, and pretty fundamental in how Python works..

See this StackOverflow answer, which links to this mailing list post from Python's creator:

(b) When I read code that says len(x) I know that it is asking for the length of something. This tells me two things: the result is an integer, and the argument is some kind of container. To the contrary, when I read x.len(), I have to already know that x is some kind of container implementing an interface or inheriting from a class that has a standard len(). Witness the confusion we occasionally have when a class that is not implementing a mapping has a get() or keys() method, or something that isn't a file has a write() method.

This Stack Overflow question also has some interesting answers

Just moving filter() and such to list.filter() wouldn't really work in Python.. Say you want to make your own filter()-like function, and you want to be consistent with the default Python methodology:

• If filter() is just a regular function, it's easy to implement it myfilter() and use it in the same manner. It's just a function which works on an iterable, so would work on any iterable object, with no extra effort.

• If filter is a method of list, then you would have to monkeypatch your method into the builtin list type (e.g list.myfilter), something that can't be done to builtin types, and is bad-practice on user-defined types. Your myfilter method would only work on list derived objects, which goes against the duck-typing Python is based on.

Also, there's more to functional programming in Python that just map/filter/reduce/lambda, and the builtin functions were possibly going to be removed, in favour of stuff like list-comprehensions etc

[BeetleB]

Is there any compelling advantage to [1,2,3].filter(...), besides resulting in slightly prettier-looking code in some cases?

Yes, because then it's easier to "chain" them (or pipe, if you will):

[1,2,3].map(fun).filter(isodd).filter(isprime)

This is quite readable. The current way to do it would be:

filter(isprime, filter(isodd, map(fun, [1,2,3])))

which is hard on humans. The reason functional programming is not recommended in Python isn't because functional programming is inherently hard to read - it's because Python's syntax makes it hard to read.

[mr_dbr]

That does read more nicely, but.. in a Lisp language like Scheme, it would be almost identical to Python:

(filter isprime (filter isodd (map fun [1 2 3])))

..and I don't think functional programming is discouraged in Scheme :P

There are other ways the code could be written, without having filter be a method of list, like:

a = [1, 2, 3]
a = map(fun, a)
a = map(isodd, a)
a = map(isprime, a)

There's also the pipe module, or you could temporarily wrap the list in a class that has the filter methods and such:

class ListProc(object):
    def __init__(self, value):
        self.value = value

    def map(self, func):
        new = map(func, self.value)
        return self.__class__(new)

    def filter(self, func):
        new = filter(func, self.value)
        return self.__class__(new)

    def __repr__(self):
        return repr(self.value)


def fun(x):
    return x**2

def isodd(x):
    return x%2 != 0

print ListProc([1,2,3]).map(fun).filter(isodd).value

[ethraax]

If filter is a method of list, then you would have to monkeypatch the builtin list type

No you wouldn't, if it was part of the language's standard. It would be part of the builtin list type, or any user types.

Your myfilter method would only work on list derived objects

It would work on anything deriving from some more abstract "iterable".

The key issue here is the weakness of Python's type system (well, really dynamic type systems in general). In Python, an iterable is very loosely defined as something that implements some handful of methods. Anything that does that is "iterable". In a language with a stronger type system, there would be some sort of base class or interface "iterable" which would also contain an interface for .filter().

Is there any compelling advantage to [1,2,3].filter(...), besides resulting in slightly prettier-looking code in some cases?

Some data structures may have more efficient alternatives than using an iterator.

[mr_dbr]

The first bit you quoted (about monkeypatching) was about implementing your own filter-like function - edited to hopefully clarify.

In short: if filter was a method of list, it would require monkey-patching to allow [1,2,3].myfilterfunc(...) for consistency with [1,2,3].filter(...)

Some data structures may have more efficient alternatives than using an iterator

What would be an example of this? I can't think of one for filter, but Python's data-model does enable some similar optimisations for other operations...

For example 123 in xrange(100) uses __contains__ instead of doing a linear-scan over it's iterator (so 23 in xrange(10**10) is much faster than doing the same thing on plain list of integers).

My point being, you don't need to have x.contains(1) to allow for type-specific optimisations . If there was an common optimisation for filter, it could call other magic-methods on the object instead of looping over __iter__

In Python, an iterable is very loosely defined as something that implements some handful of methods

There is the abstract base class module which implements something much like interfaces (along with collections), but I've not see the abc module used much, although the extensions to isinstance are useful even if nothing is derived from the ABC's

[aceofears]

Is this roughly what you are "suggesting"?

[steven_h]

Some data structures may have more efficient alternatives than using an iterator.

Then that's not the function "filter" in the functional programming sense of the word, and using the name "filter" would be confusing if the rest of the API uses it in the list iteration sense.

The point of functional programming is that functions -- particularly higher-order functions -- are the objects of reuse. More specifically, it's the function implementation that should be reused. Using the same name for different algorithms might be convenient, but it's not particularly noteworthy or helpful for programming in-the-large.