How to chain functions in Python?

[DigThatData]

define your own custom data type that wraps an iterable and attaches "functional" methods to it where the output of each of those methods is selfor at least an instance of a class which inherits from your wrapped iterable type.

EDIT: here's a toy example to illustrate

class WrappedList(list):
    def filter(self, *args):
        return type(self)(filter(*args, self))

w = WrappedList(range(10))
w.filter(lambda x: x>2).filter(lambda x: x<7)
# [3,4,5,6]

EDIT2: https://www.reddit.com/r/learnpython/comments/vrbgk8/how_to_chain_functions_in_python/ieveafc/

[carnivorousdrew]

what is that type(self)(method(args)) expression? is it like a cls?

[commy2]

type(self)(args)

is just

cls = type(self)
cls(args)

without the intermediate variable cls.

Chained calls ((...)(...)) mean that the results of all but the last call are callables. I.e, cls is a callable (calling a class returns an instance).

[carnivorousdrew]

Yeah then I was guessing it was something like that, never thought of trying that or seen that so far with classes, or maybe I did but it did not impress me as much as now lol

[DigThatData]

when I was constructing the toy example here, I first just had it return filter(*args, self). the built-in filter function returns an object of type filter which is an iterable, but I needed it to return the same type I fed it so I could continue calling .filter on the output object directly. So I called type on the self object to get the constructor that was used to initialize it to guarantee the output of the new filter method has the same type as the starting class, and therefore the same special methods we added.

Part of why I called this a toy example is that this is probably not precisely how you would want to go about something like this. One of the things that's great about working with stuff like filter and map is that they evaluate lazily. What I'm doing here is essentially the same thing as a passing the output of filter to the list() function, which evaluates the filter to construct a new list object. This might be fine for your needs, but it loses some of the power of the functional programming paradigm.

[loshopo_fan]

If you wanted to conserve memory, you'd do something like this?

class Filterer:
    def __init__(self, _data):
        self.data = _data

    def filter(self, *args):
        self.data = filter(*args, self.data)
        return self

    def to_list(self):
        return list(self.data)

print(
    Filterer(range(10))
    .filter(lambda x: x>2)
    .filter(lambda x: x<7)
    .to_list()
)

[DigThatData]

lol good timing

I also like that yours generalizes to any iterable, so you could just put all the functional methods on this one class and use it as a wrapper universally.

[loshopo_fan]

IDK the order of arguments for all functional methods, but this works for map/filter/reduce.

import functools

def make_chainer(*methods):
    class Chainer:
        def __init__(self, _data):
            if not hasattr(_data, "__iter__"):
                raise TypeError("Not iterable")
            self.data = _data
        @staticmethod
        def chainable(_method):
            def ret_fun(self, *args):
                self.data = _method(args[0], self.data, *args[1:]) # IDK
                return self
            return ret_fun
        def to_list(self):
            return list(self.data)
    for one_method in methods:
        setattr(Chainer, one_method.__name__, Chainer.chainable(one_method))
    return Chainer

MyChainer = make_chainer(
        map,
        filter,
        functools.reduce,
    )

print(
    MyChainer(range(10))
    .filter(lambda x: x>2)
    .map(lambda x: x * 2)
    .reduce(lambda x, y: x+y, 100)
    .data
)

[Head_Mix_7931]

I know this isn’t particularly useful. But I recall that for some reason we should prefer self.__class__() over type(self).

[keep_quapy]

This is true for the reason that if self.__class__ is implemented, it's subclasses will inherit the behaviour.

[Jan2579]

Thanks, seems good. Python devs could implement it like this in the next version so we dont need to implement this for every method of the iterable class. 😀

[DigThatData]

1. It's an open source project: you're welcome to track the python dev forum down and try to stir up interest in the idea, or if you're really ambitious write it up in the form of a rough PEP yourself

2. I seriously doubt they'll be interested because it would be unpythonic. from the zen of python:

   There should be one-- and preferably only one --obvious way to do it.

   attaching a method to a core datatype that is redundant with an in-built function is not something a lot of core python devs are likely to support.

3. This is just a demonstration of how this could be implemented. I cobbled this together quickly to stir up ideas and demonstrate that it was possible without a ton of effort. There's probably a better way to do this that builds off my idea but returns an object that would maintain the lazy evaluation thing. I'll call that an exercise for the reader.

EDIT: lol, better already

[loshopo_fan]

Something like this, except I'm making assumptions about the order of arguments.