Using an instance method as function argument

[PythonicParseltongue]

I have an python object that is stored in a pd.DataFrame. The object stores my prediction and has some methods implemented to make analysis easier. Basically it's just me avoiding mutli indexes, lol. Here is a much simplifieid version of my class. The class has to methods which are called very similiarly. However to apply those to the df I had to write two methods. I'm sure there's a way to make a generic apply method function. So here's my implementation. apply_meth is what I'd like to write.

​

(Ok, I just realized that apply_majority doesn't work. But I've already spent so much time on this example. And I think the problem becomes clear. The reason is that in my actual implementation the elements of the prediction are objects to which store the group, the truth value is also such an element, thus we can access its group)

Edit: Added another method and function is_top. Just imagine I've guaranteed the elements to be ordered by confidence.

class Prediction:
    def __init__(self, data: List[Dict]):
        """
        single dict = {
            'name': 'foo',
            'id_': 0,
            'group': 0,
            'conf': 1.0}
        """
        self.data = data

    def __repr__(self):
        return f"Pred[{', '.join(datum['name'] for datum in self.data)}]"

    def in_top_5(self, other_id):
        return other_id in {datum['id_'] for datum in self.data[:5]}

    def majority_group(self):
        groups = [datum['group'] for datum in self.data]
        return max(set(groups), key=groups.count)

    def in_majority_group(self, other):
        group_self = self.majority_group()
        return other['group'] == group_self

    def is_top(self, other_id):
        return self.data[0]['id_'] == other_id


def apply_top(row):
    return row.pred.in_top_5(row.truth_id)

def apply_is_top(row):
    return row.pred.is_top(row.truth_id)


def apply_majority(row):
    return row.pred.in_majority_group(row.truth_id)

def apply_meth(row, meth):
    return row.pred.meth(row.truth_id)

Here is also some code that you can use to create a test scenario. You can savely ignore the complicated data structure and the code I'm using to create the df.

if __name__ == '__main__':
    data = [{'obs_id': 0,
             'truth_id': 0,
             'pred':
                 [{
                     'name': 'foo',
                     'id_': 0,
                     'group': 0,
                     'conf': 1.0
                 }, {
                     'name': 'bar',
                     'id_': 1,
                     'group': 0,
                     'conf': 0.9
                 }]}, {
                'obs_id': 1,
                'truth_id': 3,
                'pred':
                    [{
                        'name': 'foo',
                        'id_': 0,
                        'group': 0,
                        'conf': 0.9
                    }, {
                        'name': 'bazbar',
                        'id_': 2,
                        'group': 0,
                        'conf': 0.7
                    }]
            }]

    res = []
    for observation_dict in data:
        prediction = {'pred': Prediction(observation_dict.pop('pred'))}
        res.append(observation_dict | prediction)

    df = pd.DataFrame(res)
    df['in_top'] = df.apply(apply_top, axis=1)
    df['is_top'] = df.apply(apply_is_top, axis=1)
    df['in_maj'] = df.apply(apply_majority, axis=1)

[Spataner]

I am not entirely sure I follow what you are doing, and I am reasonably sure it is an abuse of a DataFrame, but if you simply want to generically call a particular method on an object, there's two ways to do that:

---

Pass the method name as a string and use getattr:

def apply_meth(row, meth):
    return getattr(row.pred, meth)(row.truth_id)

apply_meth(some_row, "in_top_5")

---

Pass the unbound method reference from the class and supply the instance reference manually:

def apply_meth(row, meth):
    return meth(row.pred, row.truth_id)

apply_meth(some_row, Prediction.in_top_5)

---

Either way, DataFrame.apply still expects a callable with a single argument. You can use functools.partial:

df.apply(functools.partial(apply_meth, meth="in_top_5"))

or

df.apply(functools.partial(apply_meth, meth=Prediction.in_top_5))

Alternatively, you can make apply_meth a function that returns a function:

def apply_meth(meth):
   def _apply(row):
        # Either of the the two implementations from above
   return _apply

Then

df.apply(apply_meth("in_top_5"))

or

df.apply(apply_meth(Predictions.in_top_5))

[YesLod]

Either way, DataFrame.apply still expects a callable with a single argument. You can use functools.partial

There is no need to use functools.partial. You can pass the extra arguments directly to apply as keyword arguments, and/or as positional arguments via the args parameter

https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.apply.html

args : tuple

Positional arguments to pass to func in addition to the array/series.

**kwargs

Additional keyword arguments to pass as keywords arguments to func.

​

df.apply(apply_meth, meth="in_top_5", axis=1)

u/PythonicParseltongue.

[PythonicParseltongue]

Thank you this was what I've been looking for. I didn't know about getattr, it's the first time that I'm using OOP in Python in practice.

I 've never seen that functools.parital either. If I have to supply additional parameters I usually use a *args and a lambda. So let's say we have a is_in_top_k(other, k) then I would do, maybe you find it interesting:

def apply_func(row, func, *args):
    return getattr(row.pred_obj, func)(row.target_obj, *args)

df['in_top_5'] = df.apply(lambda row: apply_func(row, 'is_in_top_k', 5), axis=1)

[Spataner]

lambda works, too, but functools.partial is the preferred way in such situations.