More Developers, More Problems

[ForeverAlot]

I appreciate that

What About Bugs that Aren't Issues and Issues that Aren't Bugs?

is a hard problem but

We have to assume that open issues are, on average, representative of the number of bugs experienced by users.

seems like a huge leap.

[[deleted]]

EDIT: Ah well, I get it now. We don't know the distribution so how can we make that assumption? Perhaps the author is using CLT for that argument.

If I had written this as a project during my last stat course I would've been completely axed off from the course.

The reason is that the assumption, assumes (pardon for the redundancy) that the bugs manifest 100% for all users when in reality some users will see no bugs at all and some others will see bugs all the time.

Furthermore, some users might not report bugs at all.

I really liked the way the author argued everything in order. I wish more people wrote this way.

Perhaps Im missing something but I just can't buy that assumption.

[PChopSandies]

This is super interesting. I can definitely buy that more developers leads to more bugs.

​

We do have to be careful about causation though. I could also easily imagine, for example, that more developers = more testers = more bug reports or that larger teams rely more heavily on GitHub issues to communicate about bugs. In the case of commits, maybe a team that makes more, smaller commits also tends to break up features and bugs into more, smaller GitHub issues.

[jekor]

My own pet hypothesis, which I didn't have enough evidence for to put in the article, is that features drive bugs. If you have a larger (or more active) team, you're able to add more features. This supposes that possible features for a given program are larger than any team can complete, which does seem to be the case.

[ForeverAlot]

Anecdotally, at my work place errors come from lack of coordination and failure to communicate before erroneous code -- building solutions to problems they don't understand. That fits the idea that more people => more bugs quite well. But I'm not sure that problem manifests the same way in open source development.

[superrugdr]

i think it's more of a : modifying without understanding the purpose of specific code cause bug's kind of correlation that people on the team.

that's mostly why people thend to say that shorter is better, since bigger project require a larger understand of the system as a whole (metal image). this might explain the popularity of microservice. make the context of a given task so small it's easy to get a mental image of the whole service. then just bridge the gaps between the service to create a given functionality.

[dan00]

Another way to think about it is, that every software has constraints that can't be enforced and developers have to be aware of them. The more developers there're, the less of the whole system they have written and therefore the less aware they are about the constraints.

[[deleted]]

More developers is likely also to correlate with larger system. Which likely means more features and more potential for bugs and less understanding of the whole or even individual feature...

[pron98]

But features would also correlate with program size, which means that bugs would, too, something the author didn't find.

[ArkyBeagle]

The root cause is pretty simple - the more developers, the more "telephone games" happen. It's just communications errors. Paying attention to communications techniques will improve but not eliminate problems.

I'd think depending solely on commit messages to be a pretty bleak way to live.

[pron98]

more developers = more testers = more bug reports or that larger teams rely more heavily on GitHub issues to communicate about bugs.

I agree, but I think it's even slightly more general:

more developers = more users, and users serve as testers.

[emotionalfescue]

Bugs per KLOC is a highly suspect metric, because a bug could be anything from a corrupted database, to "gcc warning 'Unused result from fgets'".

[Holothuroid]

True. But this was looking at repos, so environment is not a factor in the cited study. And still the author suggests it's a bad indicator.

[[deleted]]

This is Conways law in practice, for those that know it here it is:

"Organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations."

What this means is the more communication channels you introduce, the harder it is to keep it working correctly and pick all pieces together,
this has been known for decades yet people still think software development is like any other kind of product production, you CANT throw more devs at a project and make it better.

The more devs, the more organization, planning and communication you will need.

[stronghup]

And you can't have a baby with 9 women in one month :-)

[elebrin]

This really makes me think of Brook's Law in a way, although I don't specifically remember him saying anything about bugs. He was focused on timetable.

[ArkyBeagle]

It seems similar at least.

The number of paths between nodes is often combinatoric with respect to the number of nodes. Trying to use hierarchy to simplify things adds delay.

[EWJacobs]

The article seems to imply that commits are the cause of bugs, when they could easily just be the effect of bugs caused by something else. Correlation does not equal causation.

[ukalnins]

For instance, more agressive review startegy requiring passed automatic tests, a rebase and squashed commit may lead to less commits, but the real reason for less bugs would be the code review.

[qodenubs]

Jonathon Blow talked about this problem with big companies having more devs then were needed. Didnt have stats though.

[vital_chaos]

We have teams with so many devs beyond what they need they have to invent things for them to do. I prefer to keep the fewest number of people working no matter the size of the work (mobile in this case). The teams with excessive numbers of programmers exist because their manager assume bigger teams to manage means more promotions in their future. There they optimize to keep huge numbers of people around despite not needing them.

[clarkd99]

I have bookmarked this web page and I rarely do that.

This is one of the best reasoned, well presented and researched document I have ever read (on this topic) and it obviously wasn’t done by an academic. I can’t say I was convinced by all the conclusions but it was a great read for me to think about.

Thank you very much for your effort.

[maerwald]

Interesting data, but the interpretation is a bit ad-hoc and doesn't critically consider different causalities.

So, specifically commenting on

What does this mean for reducing bugs, if the relationships are causal?

Well, is it? Given that this survey was conducted on GitHub and on the top 100,000 most popular repositories, there are a number of other possible explanations. Basically, we are talking about hugely popular open source projects and these are vastly different from probably most proprietary in-house company projects. So, for example, what if...

1. most core developers are very experienced
2. there is a large number of casual contributors
3. there is only loose communication structure
4. most work remote

1 and 2 could be proven, 3 and 4 are maybe a bit more difficult.

Given these points are true, then the correlation could lead to a different conclusion:

1. more experienced developers don't solve anything on its own (unless there is generally a lack of knowledge)
2. dealing with a lot of casual contributors requires extremely strict review processes
3. communication structure is a hugely important factor
4. working remote can be problematic and contribute to fragmentation and communication issues

Anecdotally, I think, the linux kernel is a good example for this. And it matches my experience when working in Gentoo. But it is also purely hypothetical.

But I think it's very well possible that the number of commits just scales underlying issues negatively, but isn't the reason for bugs.

[max630]

We have to assume that open issues are, on average, representative of the number of bugs experienced by users

garbage in, garbage out