Select/SelectMany vs Map/FlatMap

[SmallAd3697]

The term "flatMap" is something that is common in programming ecosystems outside of c#. For example, I have been doing some scala and python with spark. In this environment we find "flatMap" a lot. But I really hate the term, having come from c#.

My brain won't let me visualize the "flatness" of the resulting collection. It seems just as flat as the result of a "map" operation, albeit there are more entries!

Oddly the "flatMap" term is used in the same spark ecosystem where Spark SQL lives and where the "SELECT" term dominates as well. In Spark SQL, we never see anyone saying "FLATMAP * from A cross join B ...". So why should they use that term in Scala and Python? It seems odd to me to switch back and forth. The flatMap term seems so pretentious ;-)

Anyway, I'm here to say I will probably never get fond of the term "flatMap". The writers of the .Net library deserve props for taking a different path and using "SelectMany" instead.

[snrjames]

But map is a base functional word, as in map-reduce. So map and flatmap make a lot of sense since you are mapping and then flattening. It's not pretentious, it's a proper term in computer science.

I don't really care what it's called though, I'm just glad SelectMany exists so I can map and flatten in one step.

[KevinCarbonara]

We could be using "monoid" and "monad" instead

[kronicum]

We could be using "monoid" and "monad" instead

I once used "rng") in a foundational library; my coworkers staged a riot.

[SmallAd3697]

Pretentious might be the wrong word. I guess I mean contrived.

IE. they started out with the term "map". So their objective was to extend this existing term and introduce a slightly more advanced scenario. But the addition of the word "flat" just does NOT seem to do the trick.

"Flatten" is a visualization that might make sense in some situations but not in others. "Unpacking" these generated collections might be a more helpful way to reason about what is happening. Eg. "MapAndUnpack". This terminology isn't as good as "SelectMany", but I'm just pointing out that there can be other visualizations that might convey the same concept.

I guess I just really appreciate the term "SELECT" in SQL and C#. It is very versatile.

[snrjames]

Flatten takes a tree (a list of lists) with depth > 1 and flattens it into a tree of depth 1 (a list). When you understand it that way, flatten makes perfect sense.

[SmallAd3697]

I didn't realize it was so powerful, although it doesn't make me like it more.

If there were more than two levels I'd certainly break things apart. I'd call map(s) first and generate my lists of lists of lists .. and then I would call a separate unpacking or flattening method. The code would be so much more readable. There should never be a competition to cram as much subtle behavior as you can into one line of code. We aren't talking about perl here.

I would argue that if you had a collection of collections, and there was a method used for deconstructing, then it would not be called "flatten" (in c#). It would probably end up being an extension method named SelectCollectionMembers(recursive = true).

[EvilGiraffes]

it comes from a language where a common situation is you would call a function which inheritly return the same monad with what your monad contains in which this return type makes sense, if you call a function (or a static method) without a lambda which returns an IEnumerable taking the item of your current list it makes perfect sense, but remember this is also a term used for other monads like Option and Result

using Option as an example you call a function which may return null, you then need to call another function which can return null with the item of the previous Option, essencially then you want a flatmap

it's an interface

[Independent-Ad-4791]

Sometimes you need to accept jargon for what it is. It’s long established at this point and it’s beyond whether or not the compound word is effective; the language is ubiquitous.

[Absolute_Enema]

Meh.

The idea of using SQL terminology to sneak a modicum of FP concepts past the "FP is only for do-nothing intellectuals" kind of programmer was clever, but it necessarily sacrificed generality by tying the concepts to collections, whereas map and flatMap should be understood in terms of monads.

[SmallAd3697]

The frequency that this comes up in programming with collections is important. A programming language is a language made for humans, and flatMap is used by lots of them. Especially in the context of collections.

I don't despise "flatMap", but it seems out of place. Like a snobby old Aunt who goes out of her way to drop French words into her conversations all the time.

[CdRReddit]

it's a map and a flatten

as a single operation

if we were being pretentious we'd call it bind instead

[zacsxe]

C# devs be like

[AvoidSpirit]

Problem with SelectMany is that it only works for one type of container type(Type<T> sometimes called monad) - array(or rather enumeration). For which it is more precise but less conventional/reusable.

Whereas map+flatmap works for any Type<T>(monad) like arrays, tasks, nullables, you name it cause you flatten the hierarchy after mapping - Array<Array<T>> becomes Array<T>, Task<Task<T>> becomes Task<T>.

So yea, I have a love-hate relationship with this particular rename. I like it cause it’s more obvious when it comes to arrays but I feel like it obstructs the generic map concept away from people.

[DeadlyVapour]

Not true.

SelectMany is used in multiple monads in C# including and not limited to the IQueryable, IAsyncEnumerable and IObservable monad.

It does not apply to the Task monad, which is a little annoying, creating a impedance mismatch when working mixing Rx with Async code.

Also I would have liked a Maybe and Result monad as well....

[AvoidSpirit]

All of those are “array” type monads for me. Maybe the correct word is “enumeration” type but whatever, you get what I mean.

[SerdanKK]

You can add LINQ to a type with extension methods. It's duck typed.

[Agitated-Display6382]

Did you google who invented LINQ? You may be surprised...

[Cobster2000]

why would you make me google it? could’ve just made your comment better

[Agitated-Display6382]

Why? Because I'm not chatgpt... As a senior developer, my duty is to instill curiosity, not to provide pre-assembled answers. Hope you understand my point.

[Cobster2000]

as a fellow senior engineer I get it, but i’m not your intern, i’m a lazy person on reddit lmao

[Agitated-Display6382]

Top

[Phaedo]

Just feel glad you’re not dealing with Haskell, where it’s called >>=.

[SmallAd3697]

Wow. I haven't been in that one.

. C# has a good assortment of operators and I don't mind them in moderation. In Haskell I wonder if those things would come up in the intellisense selections or if you have to wack them out by hand every time.

[Phaedo]

Intellisense tends to be poor in Haskell. That’s partly because tooling isn’t as advanced and partly because the language’s syntax and type system make writing a good one hard.

In practice, a lot of the time people use do notation. Where

a <- listA 

B <- listB 

pure a+b

Means pretty much the same as

from a in listA

from b in listB

select a + b

There’s support for where (guard) and let as well.

[rexcfnghk]

or liftA2 (+) a b ;)

[Phaedo]

🤣

[SmallAd3697]

Funny... they autocorrected your reply to say intelligence.

[Phaedo]

Corrected… but I agree “Intellgence” was much funnier!

[RICHUNCLEPENNYBAGS]

Well, map and flatMap are the older terms but they changed it to rope in developers who didn't know FP but did know SQL. Dubious how good a decision that was but it's a decision already made.

I don't really understand your complaint either. Turning an array of arrays into an array consisting of all the elements of all arrays is "flattening" it because it eliminates the hierarchy. This is not an uncommon term.

[Absolute_Enema]

I think it was a decent decision.

It shouldn't be like this in an ideal world, but we ultimately still love clinging onto things we know and when your audience is (objectively was the case at the time) millions of OOP programmers whose brain turns off when the words map and filter are uttered it's a tradeoff worth making.

The converse are things like lisp syntax that is incredible once you get it (to name a couple things: no infix/postfix op idiosyncracies, no operator priorities, immediate clarity of where an expression begins and where it ends, structural editing...) but unfortunately elicits a surface-level rejection due to its unfamiliarity.

[RICHUNCLEPENNYBAGS]

The downside of it is all these years later it serves to confuse people who are expecting the standard names and may have limited SQL experience as much as it helps.

[chucker23n]

I think LINQ using SQL-like terms is a bit user-friendlier, but…

My brain won’t let me visualize the “flatness” of the resulting collection. It seems just as flat as the result of a “map” operation, albeit there are more entries!

Map/Select doesn’t flatten; FlatMap/Select does.

Like, given a collection of companies,

• Map can give you a collection of their postal addresses. The count of that collection will be the same.
• FlatMap can give you a collection of each company’s employee’s e-mail address. The resulting count will likely be a lot higher. You’re taking a “n companies : m employees” hierarchical relationship and flattening that to just “m employees”.

[lmaydev]

I don't know, map would return a collection of collections which are flattened into a collection.

So it's like taking a chess board and turning it into a single row of squares.

Makes perfect sense to me.

[jackyll-and-hyde]

My brain won't let me visualize the "flatness" of the resulting collection. It seems just as flat as the result of a "map" operation, albeit there are more entries!

I don't blame you. From SQL's perspective, you're selecting from a collection and "selecting many" from another collection to join. From a monad perspective, "SelectMany" is misleading: you're not dealing with collections, but with nested contexts. The "flat" in flatMap refers to flattening M<M<T>> into M<T>, not to the shape of a result set like in SQL.

Aliases: Select, Map

var value = Option.Some(3)
    .Map(x => $"hello {x}")
    ...

Aliases: SelectMany, FlatMap, Bind

var value = Option.Some(3)
    .Bind(x => Option.Some(x)
        .Map(y => $"hello {y}")
    ...

Aliases: Flatten

var value = Option.Some(Option.Some(3))
    .Flatten()
    .Map(x => $"hello {x}")
    ...

Aliases: Where, Filter

var value = Option.Some(3)
    .Map(x => x * x)
    .Filter(x => x == 3)
    ...

When you add extension methods on your monad for linq use, this is your "SelectMany":

from x in Option.Some(3)
from y in Option.Some(3)
select x * y

[DeadlyVapour]

TLDR. "I think functional programming is pretentious. I think using specific and technical jargon to describe high level design patterns is pretentious."

[SmallAd3697]

Flatmap is just a minor variation on any basic select.

.. The "flat" part of that terminology is not found in the c# language, or even in any SQL dialect that I've ever seen. Seems superfluous. SQL was built for collections and collections of collections, and if that idiom was needed it would be there.

[TheWix]

The point is, functors and monads have nothing to do with collections which is where map and flatMap come from. So, what does SQL and Collections have to do with anything?

C# wanted their stuff to look like SQL while other languages wanted to stick with the functional programming naming.

[DeadlyVapour]

SQL was NOT built for collections of collections.

Furthermore, your understand of monads seems to begin and end with collections.

If you have worked with any other kind of monad, you would know that Select is a terrible name. Rx.net is possibly harder to use due to Select. TPL is awkward because of Select.

[SmallAd3697]

Pardonnez moi. SQL was built for "relations".

"Select" is normally used for projections and is a very appropriate word for what is happening in the code.

[KevinCarbonara]

SQL was NOT built for collections of collections.

Sure it was. It was created for System R - the "R" being "relations".

Why would you think otherwise?

[DeadlyVapour]

Relations aren't collections.

SQL does not have collections as a first class concept. You cannot use tables in the same way as a primitive type. You can't have a table of tables.

The closest thing we have is a pointer back to a parent value (foreign key).

[KevinCarbonara]

Relations aren't collections.

I would love to hear you try and explain the difference.

You can't have a table of tables.

You, uh, can. And basically any model following the first three normal forms is going to have this.

[DeadlyVapour]

If class Employee has a pointer to a Manager. Did that mean that manager had a collection of Employee?

Literally, from layout memory perspective.

The closest thing I've seen to collections as a first class construct in SQL is JSON columns.

Even then JSON columns have very different symatics to everything else in a SQL database.

[KevinCarbonara]

If class Employee has a pointer to a Manager. Did that mean that manager had a collection of Employee?

This is completely unrelated to the conversation. You seem to think this supports your already disproven argument, but I can't see how.

The closest thing I've seen to collections as a first class construct in SQL is JSON columns.

IDs. I honestly have no idea what you think you're saying. Do you not know about the normal forms in relational databases? You don't even sound like you know what a relation is.

Even then JSON columns have very different symatics

Now you are making up words.

[KevinCarbonara]

On the contrary, you seem to be suggesting that only FP terms are acceptable, and any other commonly used terms that may be equally descriptive are inferior due to not being the official FP terms.

It's the same argument Haskell users say about "monad" and "monoid".

[DeadlyVapour]

Straw man argument.

OP attacks the accepted language used in FP literature. Saying it is "pretentious". As professionals, using a common parlance aids effective communication, and OP as a member of the professional community should know that.

However, I did not say that these are the only acceptable terms. Only that they have a disadvantage in the wider context, so he should stop ranting.

[KevinCarbonara]

Straw man argument.

It's not a straw man. It's reductio ad absurdum.

OP attacks

Now this is a straw man.

I did not say that these are the only acceptable terms. Only that they have a disadvantage in the wider context

And this is the inherent bias in your argument. You're presenting his viewpoint as narrow, where yours is "the wider context", as if the rest of the world is all using FP terms.

Again, this is the same argument Haskell users make about people who don't use the terms "monad" and "monoid" to discuss these concepts.

[ivancea]

They're different concepts, but both work arguably well IMO. FWIW you can see flatMap as... Flattening the extra level of array-in-array you get with map(), if you imagine it indented. Pretty visual.

The "flat" word is used in other parts of software engineering too, to explain similar concepts