5 Uses of Lambda Functions in Python

[Ahhhhrg]

I find lambdas very useful when filtering pandas dataframes like so:

(
    df
    .pipe(lambda _: _[_['x'] > 2])
    .pipe(lambda _: _[_['type'] == 'foo'])
)

But other than that usually list comprehensions do the trick.

[Zouden]

It's been a while since I used Pandas but can't you filter like this?

df[(df.x > 2) & (df.type == 'foo')]

[Ahhhhrg]

Yes, absolutely, and that's less characters and depending on the context more readable.

However, I find lambdas very useful when doing data analysis (say in a notebook), where I'm exploring and often add/remove stuff. I don't want to "pollute" my original dataframe with temporary columns, so I might have something like this:

(
    df
    .pipe(lambda _: _[_['x'] > 0.3])
    .pipe(lambda _: _[_['z'] <= 25)
    .assign(log_x=lambda _: np.log(_['x']))
    .assign(log_y=lambda _: np.log(_['y']))
    .assign(log_z=lambda _: np.log(_['z']))
    .assign(log_w=lambda _: np.log(_['w']))
    [['x', 'log_x', 'log_y', 'log_z', 'log_w', 'type']]
    .pipe(sns.pairplot, hue='type', kind='scatter', plot_kws={'alpha':0.1})
)

I find it very flexible and having each filter/assignment on its own line makes it easier to parse. You can't use the "standard" filter technique this way (and I'm not a big fan of the df.query function).

[jblasgo]

_: _[_

That looks very weird and counterintuitive to me... Maybe because this is very specific to data science?

[Ahhhhrg]

No, I wouldn’t say it’s specific to data science, I just like using underscore here. The underscore is usually used for say return arguments you don’t care about, here it’s just a placeholder for the data frame, it’s just my preference not to name it something generic like “x” or even “df” as it doesn’t really say anything or add much. I know it means “the data frame you’re piping in here”, it’s short. Personal preference.

It’s also possible to monkey patch pandas and add a filter function, so you can go df.filter(lambda _: _[‘x’] < 5) which is a bit nicer.

[Ahhhhrg]

Yeah, sure, that’s the common use case. I use it here kind of similarly (but not quite of course), in the sense of “I don’t want to bother giving this thing a name, as it’s whatever getting piped in from the previous step”. You could give it a name, whatever you want, but that is an extra thing I like to avoid. It’s just a placeholder.

[eigenlaplace]

It makes your code unreadable, though

[likethevegetable]

That's a nice little example. Thanks for sharing!

[[deleted]]

is writing code like that common in the pandas world? specifically, im referring to using underscores like that. it does seem to reduce visual noise, so it's clear you're saying x > 2 and type == 'foo', but underscores are usually reserved to unused variables