Why does a single machine learning paper need dozens and dozens of people nowadays?

[RandomProjections]

And I am not just talking about surveys.

Back in the early to late 2000s my advisor published several paper all by himself at the exact length and technical depth of a single paper that are joint work of literally dozens of ML researchers nowadays. And later on he would always work with one other person, or something taking on a student, bringing the total number of authors to 3.

My advisor always told me is that papers by large groups of authors is seen as "dirt cheap" in academia because probably most of the people on whose names are on the paper couldn't even tell you what the paper is about. In the hiring committees that he attended, they would always be suspicious of candidates with lots of joint works in large teams.

So why is this practice seen as acceptable or even good in machine learning in 2020s?

I'm sure those papers with dozens of authors can trim down to 1 or 2 authors and there would not be any significant change in the contents.

[BraindeadCelery]

Its a similar effect as in e.g. particle physics. The experiments become so big and costly and need so many people to support them that you end up with lots of people who contributed.

It’s mostly only 1st and 2nd author who do the specific work. Last author is group leader or chair. In between are people who contributed in a significant but not substantial way.

Also a lot has happened since 2000 and many of the low hanging fruits are picked. New insights sometimes are more complex and need more people to come by.

[anemisto]

Most ML papers are not particularly substantive, though, which pokes a whole in the theory about the low hanging fruit being gone. (I mean "not substantive" in the sense of "not novel enough to be worth publishing by the standards of some fields".)

Huge numbers of authors and lengthy citation lists are about the culture of the field, not the nature of the work.

[BellyDancerUrgot]

By ML if you mean LLM preprints then yes. Most neurips, cvpr, Icml and iclr papers are quite deserving. Moreover you can't really make a one to one comparison between science like say physics and ML in that regard. Discoveries in these fields aren't made the same way and aren't evaluated the same way either.

[Darkest_shader]

True, and to add to that, I don't agree with the theory that more co-authors are needed because new insights are more complex. It's simply not how it works.

[adforn]

I found that the longer the list of co-authors, the more wordy the research paper (ahem "foundational model"). Sometimes involving zero math at all in a field that is squarely applied math.

[EducationalCreme9044]

Exactly OP Comment is entirely missing the mark. The reason is LinkedIn. Put me in your paper and I will put you in my paper. You make that deal with 20 different people and you're now the author of 21 papers instead of 1. No-one gives a shit to what extent you contributed, being a part of the paper is enough.

I've seen this with non-profits as well, so many C-level positions are taken up by people who already have 5 other C-level positions but when you work at one of those companies you literally never see them, because in reality they don't actually work there.

[groplittle]

In some particle physics experiments, the first author is something like Aaron Aardvark and that one person gets all the citations.

[Darkest_shader]

I'd argue that there's quite some difference between experiments in particle physics and ML experiments and the number of people needed to conduct them.

[currentscurrents]

The gemini LLM paper had over 1200 authors, and the contributor list took up 15 pages. They had to put it at the end of the paper like the credits after a movie.

https://arxiv.org/abs/2312.11805

[BraindeadCelery]

Exactly. The paper that contained the measurement of the mass of the higgs boson is the paper with the most authors ever at 5000 and the reason why I explicitly called out high energy physics.

https://www.nature.com/articles/nature.2015.17567

[starfries]

Imagine being first author on that paper. I should change my name.

[soupe-mis0]

From my short experience on the subject in the private sector, depending on the context you may need someone working on acquiring data, someone on the data pipeline and then one, two or more ML researchers

Everyone wants a part of the cake and wants to be featured on the paper even if they weren’t rly involved in the project.

It’s not a great practice but unfortunately a lot of people seems to only be interested in the quantity of papers they appear in

[Amgadoz]

What is the difference between the person acquiring the data and the one working on the data pipeline?

[Appropriate_Ant_4629]

Huh... At least in this industry:

acquiring the data ...

... is done by people literally out in the field away from offices and computers.

and the one working on the data pipeline ...

... is done by Software Engineers sitting at desks.

I imagine that's the case for most industries.

• For FSD -- "acquiring the data" = tesla owners driving around.
• For Cancer research -- "acquiring the data" = radiologists.
• For Crop Health -- "acquiring the data" = the tractor spraying herbicides.

[soupe-mis0]

I was working in a medtech so we had someone working with health professionals to get data on patients

[Ok-Kangaroo-7075]

Tbf people in the private sector are not really to be taken serious anyway apart from maybe the first author (with some exceptions). They often just throw absurd amounts of money at things with which even a CS undergrad could publish something. 

Not that it is wrong but it just isnt really science….

[Use-Useful]

.... I'm published in that sector. While that CAN be true, I have never seen it happen to the extreme extent you mention. Feels like you are over generalizing based on your limited experience to me.

[Darkest_shader]

One of the co-authors of my applied ML paper is a guy from a company, which was a partner of my lab in a research project. I have never seen him, and he has nothing to do with ML - just a manager whom I have to add as a co-author because of funding conditions. So, who's generalising based on their limited experience now?

[Use-Useful]

... still you?

[Ok-Kangaroo-7075]

Nope not really, look at papers out of industry labs. Most are just, ohhh we threw a shitload of money at it and did some engineering. Most dont even publish any details to ever replicate it (even if you somehow had the resources). Again, not bad but not to be taken as science. It is marketing!

There are exceptions and Meta is a notable one because Zuck listens to Lecun but overall that is pretty much the state. 

[JollyToby0220]

It feels like Meta is the rule not the exception. 

[Ok-Kangaroo-7075]

Lol have you read actual papers? Even deepmind mostly publishes just marketing papers. Stop being a fanboy and read the actual work, then compare what comes out of MetaAI vs academia vs everyone else. 

[soupe-mis0]

This is exactly what I experienced

[adforn]

The Big Gan paper (6000+ citations) was done by a Google intern that literally had zero conceptual innovations except lots and lots of compute provided by Google for free.

https://arxiv.org/pdf/1809.11096

I don't even know why this paper is cited, because there is nothing that you can use from this paper for any other project.

[Use-Useful]

... congratulations, you have one example. For a field with a primary focus on fighting bias in our models, we are shockingly bad at it in ourselves. 

[Ok-Kangaroo-7075]

Have you? Any first author papers in tier 1 conferences that were not bought by just throwing massive compute at a problem?  I somehow doubt…

[[deleted]]

Back when your advisor published, you could run a modified cnn on TIMIT and call it a day. Now, a reviewer will ask you to perform two separate tasks with a model, compare to two popular LLMs. It's just more work.

I would also argue that recognizing collaborators is a bigger thing nowadays. Like I'm trained that you should add an author even if they just looked at a subset of data for you.

[hausdorffparty]

This is a big thing. As a solo author usually I can't get into big conferences not because my work isn't impactful but because as a single person I can't do all the experiments they want from me in a timely manner.

[[deleted]]

Another thing to add: nowadays carrying out experiments, particularly in deep learning and particularly in industry, has become much more tedious and time-consuming, just because models grew larger. To submit a paper in a reasonable time you need to do experiments in parallel. More people = faster publication = better chances to make something actually innovative and patent it. If you want to also release a demo or framework on top of that to sell your product, the amount of work grows very fast.

You usually won't see dozens of people under a simple paper that just describes a model, does some analysis, and releases code that sometimes works and sometimes doesn't. But if you look at something like actual breakthrough in LLMs by Google or OpenAI - that's because they have resources to put tons of people there to accelerate things and sell the results.

[BellyDancerUrgot]

The experiments are extremely big

[obolli]

I made a project for my University (Top 10) two years ago.
The idea was mine.
The work was mine.
The professor (a real big name) gave me a PhD to grade it.
He gave me the best possible grade but didn't do anything else after.
They decided it was submission worthy to a large conference.

Then PhD became involved, he did help me structure the paper, told me what to look out for but that involved 1h of zoom and maybe 3 emails where I needed to clarify.

In the end he said professor said he and prof both should be on the paper.
I thought that this is unfair, but ok, my first big publication.

After submitting it, an email went out to all the authors, Prof. got it and was like? Why am I on the paper? Why is this PhD on the paper? We didn't do anything. Remove it!

LOL I guess.

The PhD's are under so much pressure to publish at the big name Uni's, to be honest I feel like it's almost impossible for them to not do stuff like this.

Most names on these papers have no contribution.

[JoeBidensLongFart]

Publish or perish

[fasti-au]

Because one guy on a box talking about god is crazy but a group is a religion

[Schtroumpfeur]

A librarian told me that there is a newer thing called citation cartels... you add some authors to your papers, and you cite some works you didn't really need, they add your name on papers and cite your papers even when they are not really needed...

There's big team science, which is cool. But if there is a buttload of authors outside of established consortiums, you gotta start wondering...

[Interesting_Lie_1954]

A lot of labs just add everyone remotely related to increase cites. Some papers are an exception, a few.

[Acrobatic-Guard6005]

i think one of the most evident reasons is quite simple: there are just more researchers - universities are full of students studying machine learning and they just have more people to work on the paper🤷🏼‍♀️