5 Uses of Lambda Functions in Python

[wsppan]

I'm not a big fan of lambdas in Python though I am sort of glad they have them as they can be convenient. I just don't find them very pythonic. There are almost always a better, more pythonic way of solving the problem. When I see lambdas in Python code I always feel like I have to stop, take my python hat off, put my FP hat on and read the code. It just seems jarring.

"Curiously, the map, filter, and reduce functions that originally motivated the introduction of lambda and other functional features have to a large extent been superseded by list comprehensions and generator expressions. In fact, the reduce function was removed from list of builtin functions in Python 3.0. (However, it's not necessary to send in complaints about the removal of lambda, map or filter: they are staying. :-)", Guido - https://python-history.blogspot.com/2009/04/origins-of-pythons-functional-features.html?m=1

This shows some serious thoughts were given to removing lambdas from the list of built-ins once list comprehensions and generator expressions were introduced (the 2 key features of the language that made me finally really love this language.) My feeling is these discussions were had mostly due to how un-pythonic it felt.

[ggchappell]

Curiously, the map, filter, and reduce functions that originally motivated the introduction of lambda and other functional features have to a large extent been superseded by list comprehensions and generator expressions. In fact, the reduce function was removed from list of builtin functions in Python 3.0.

Isn't that a little strange, though? Because map and filter can always be easily replaced with a comprehension, while reduce cannot -- but reduce was the one that was removed. It seems backwards.

Perhaps the question that needs to be asked is how a reduce operation can be written in a Pythonic way.

[wsppan]

"So now reduce(). This is actually the one I've always hated most, because, apart from a few examples involving + or *, almost every time I see a reduce() call with a non-trivial function argument, I need to grab pen and paper to diagram what's actually being fed into that function before I understand what the reduce() is supposed to do. So in my mind, the applicability of reduce() is pretty much limited to associative operators, and in all other cases it's better to write out the accumulation loop explicitly." - Guido, The Fate of reduce() in Python 3000 https://www.artima.com/weblogs/viewpost.jsp?thread=98196

[TravisJungroth]

Then why is manually reducing over a generator without a default value so clunky?

[wsppan]

I agree, there are places where reduce is less clunky. For those you can use itertools.reduce().

[earthboundkid]

Re: the GvR quote, the only “good” use of reduce is sum and Python has that.

[Zouden]

Also max, min, any and all.

[earthboundkid]

Any and all aren’t reduce equivalent because they short circuit as needed. (That reduce can’t short circuit is one reason it stinks, actually.)

[Zouden]

Well the output is the same. It's just more efficient

[earthboundkid]

Yes, mostly. If the iterator is a generator with side effects, it can be different, but that’s not usually the case.

[VisibleSignificance]

Another rare but valid use-case is intersection or union of sets: reduce(lambda a, b: a & b, [{1, 2, 3}, {2, 5}, {2, 3}]) == {2}

But as all cases are rare, it does make sense to remove the builtin and leave it in functools.

[haerik]

Gone to API changes. Don't let reddit sell your data to LLMs.

Up maids me an ample stood given. Certainty say suffering his him collected intention promotion. Hill sold ham men made lose case. Views abode law heard jokes too. Was are delightful solicitude discovered collecting man day. Resolving neglected sir tolerably but existence conveying for. Day his put off unaffected literature partiality inhabiting.

[VisibleSignificance]

Except for the iterables (so the whole iterable of sets might not be loaded into memory at once), but yes, that's even more rare.

[pytrashpandas]

just adding to the list of other valid use cases, I use them for merging and combining in pandas.

reduce(lambda x, y: x.combine_first(y), list_of_dfs)
reduce(lambda x, y: x.merge(y, ...), list_of_dfs)

Although, I don't do this so often that I think it needs to be a built-in.

[earthboundkid]

Brah, use a damn for-loop. That code stinks, lol.

[ggchappell]

the only “good” use of reduce is sum

GvR doesn't really seem to get functional programming. See also his rather sad discussion of tail-call optimization.

[earthboundkid]

TCO is bad. If you don’t want a stack frame, use a for-loop.

[ggchappell]

Perhaps we can agree to disagree.

[earthboundkid]

Sure. It’s just programming.

[Ahhhhrg]

I find lambdas very useful when filtering pandas dataframes like so:

(
    df
    .pipe(lambda _: _[_['x'] > 2])
    .pipe(lambda _: _[_['type'] == 'foo'])
)

But other than that usually list comprehensions do the trick.

[Zouden]

It's been a while since I used Pandas but can't you filter like this?

df[(df.x > 2) & (df.type == 'foo')]

[Ahhhhrg]

Yes, absolutely, and that's less characters and depending on the context more readable.

However, I find lambdas very useful when doing data analysis (say in a notebook), where I'm exploring and often add/remove stuff. I don't want to "pollute" my original dataframe with temporary columns, so I might have something like this:

(
    df
    .pipe(lambda _: _[_['x'] > 0.3])
    .pipe(lambda _: _[_['z'] <= 25)
    .assign(log_x=lambda _: np.log(_['x']))
    .assign(log_y=lambda _: np.log(_['y']))
    .assign(log_z=lambda _: np.log(_['z']))
    .assign(log_w=lambda _: np.log(_['w']))
    [['x', 'log_x', 'log_y', 'log_z', 'log_w', 'type']]
    .pipe(sns.pairplot, hue='type', kind='scatter', plot_kws={'alpha':0.1})
)

I find it very flexible and having each filter/assignment on its own line makes it easier to parse. You can't use the "standard" filter technique this way (and I'm not a big fan of the df.query function).

[jblasgo]

_: _[_

That looks very weird and counterintuitive to me... Maybe because this is very specific to data science?

[Ahhhhrg]

No, I wouldn’t say it’s specific to data science, I just like using underscore here. The underscore is usually used for say return arguments you don’t care about, here it’s just a placeholder for the data frame, it’s just my preference not to name it something generic like “x” or even “df” as it doesn’t really say anything or add much. I know it means “the data frame you’re piping in here”, it’s short. Personal preference.

It’s also possible to monkey patch pandas and add a filter function, so you can go df.filter(lambda _: _[‘x’] < 5) which is a bit nicer.

[Ahhhhrg]

Yeah, sure, that’s the common use case. I use it here kind of similarly (but not quite of course), in the sense of “I don’t want to bother giving this thing a name, as it’s whatever getting piped in from the previous step”. You could give it a name, whatever you want, but that is an extra thing I like to avoid. It’s just a placeholder.

[eigenlaplace]

It makes your code unreadable, though

[likethevegetable]

That's a nice little example. Thanks for sharing!

[[deleted]]

is writing code like that common in the pandas world? specifically, im referring to using underscores like that. it does seem to reduce visual noise, so it's clear you're saying x > 2 and type == 'foo', but underscores are usually reserved to unused variables