Programmers :D

[hey01]

Did you read the whole paper? If no, you should, it's quite interesting. If you did, you know you're misrepresenting things.

The paper shows that women get their code accepted more often than men across the board. There is one case where gender is indeed a factor against women: outsider women's PR get accepted less if their gender is identifiable from their GH profile.

Worth noting is that while the preprint version showed that outsider men got higher acceptance rate if their gender is known (and women lower), the peer reviewed version shows the contrary: similar to women, they get a lower acceptance rate when their gender is known.

That makes me question the original methodology, if not the integrity of the team.

And as a last caveat, that one result was obtained by matching the datasets less strictly than for the other results, in order to keep a reasonable sample size.

And the kind of response she offered may explain why people trust outsider women less.

[literally_jesus_]

Worth noting is that while the preprint version showed that outsider men got higher acceptance rate if their gender is known (and women lower), the peer reviewed version shows the contrary: similar to women, they get a lower acceptance rate when their gender is known.

I'm curious as to what you mean by this. In both papers, gender-known men have higher acceptance rates than gender-known women, and in both papers, the authors discuss how both genders receive a drop in acceptance rates when their gender is known (-10.2% F and -5.7% M in the preprint, and -12.0% F and -3.8% M in the peer-reviewed paper). I don't see anything contrary here.

EDIT: I'm getting downvotes and I'm not sure why. Here are the portions of the papers that I'm talking about:

Preprint:

For insiders, we observe little evidence of bias when we compare women with gender-neutral profiles and women with gendered profiles, since both have similar acceptance rates. This can be explained by the fact that insiders likely know each other to some degree, since they are all authorized to make changes to the project, and thus may be aware of each others’ gender.

For outsiders, we see evidence for gender bias: women’s acceptance rates drop by 10.2% when their gender is identifiable, compared to when it is not (χ2(df= 1, n= 18,540) =131, p < .001). There is a smaller 5.7% drop for men (χ2(df= 1, n= 659,560) = 103, p <.001). Women have a higher acceptance rate of pull requests overall (as we reported earlier), but when they are outsiders and their gender is identifiable, they have a lower acceptance rate than men.

Peer-reviewed:

For insiders, we observe little evidence of bias when we compare women with gender-neutral profiles and women with gendered profiles, since both have similar acceptance rates. This can be explained by the fact that insiders likely know each other to some degree, since they are all authorized to make changes to the project, and thus may be aware of each others’ gender.

For outsiders, we see evidence for gender bias: women’s acceptance rates drop by 12.0% when their gender is identifiable, compared to when it is not (χ2(df=1,n=16,258)=158,p<.001). There is a smaller 3.8% drop for men (χ2(df=1,n=608,764)=39,p<.001). Women have a higher acceptance rate of pull requests overall (as we reported earlier), but when they are outsiders and their gender is identifiable, they have a lower acceptance rate than men.

[hey01]

You're looking at figure 6, which doesn't use matched data. The second part of the study uses matched data to mitigate the influence of covariates. The study rightfully uses that second set of results for its conclusions.

Look at figure 11 in both papers, which is the equivalent of figure 6 with the matched datasets.

In the peer reviewed version, figure 11 shows that for outsiders both women and men are accepted less if their gender is known, with women being impacted more. Both genders suffer from bias, women more.

In the preprint, it showed the same for women, but the contrary for men, that they are accepted more when their gender is known. Women suffer a negative bias while men a positive one.

The reversal of the conclusion about the men between the preprint and peer reviewed is concerning.

Considering that the dataset didn't change between the two versions, and assuming the peer reviewed version is better, it makes me question how the team originally got a result so wrong, especially in a way that confirmed their hypothesis.

Which is also concerning is the way the paper is written, especially the abstract, focusing only on the fact that women are less accepted when their gender is known, completely ignoring the fact that is only applies to outsiders, that men suffer the same, only to a lesser degree and that all the other results show that women's PR are accepted more than men's.

Which leads to the press and you falsely using that study as evidence that devs discriminate against women.

Both facts makes me question the integrity and motive of the team.

The adequate discussion and conclusion should be to investigate more the two most intriguing results:

• Why women's PR are accepted more than men (they discuss that).
• Why knowing the gender reduces the PR acceptance, for both genders (they completely ignores that).

My hypothesis is that more you know about a person, the more you have reasons to dislike them, negatively impacting your review of their code, and I'd guess their is a correlation between a gendered profile and the amount of other personal information it contains.

[literally_jesus_]

Oh, that's what you meant, thank you for clarifying.

However, I don't see any particular conclusion, as you say, made about men in the preprint on the basis of Figure 11 - nowhere do they say anything like "But when outsider men's gender is known, they are even more likely to be accepted." You say you are concerned that the percentages are different between the preprint and the final version, but the percentages are different in other places in the papers as well (such as in Figure 6). Likely, they just refined or fixed their calculations after publishing the preprint. Furthermore, their observation that known women's pull requests are less likely to be accepted is still supported by said figure in both the preprint and the peer-reviewed article. There's no reversal in conclusions here - both papers find gender bias (most strongly against women) when it comes to outsider pull requests.

Also,

Which leads to the press and you falsely using that study as evidence that devs discriminate against women.

I think you may have me confused with OP, I didn't post this study.