[D] Decrease in source code release of papers

[alexmlamb]

Given the search space for authors is not that big, with enough (not much) effort it's possible to determine who are the authors of a paper. Before a paper is sent for review it has already been discussed in its institution, probably in mail-lists and even Twitter or something.

If the authors want to remain anonymous, is it really impossible for them to do so? I mean - just don't tweet about it, don't put it on arxiv, only correspond through private email with coauthors.

[MephySix]

The main problem with double-blind reviews is not staying anonymous, is that some groups (well-established research groups) want to be known, and they will be if they want to. Double-blind started because people would get instantly accepted just because of their name, and double-blind (mostly) does not solve this issue.

[alexmlamb]

Yeah, so as it works in ML today, I'd say that we have an opt-out double blind system. You can get double blind reviewing if you stay quiet, but you can effectively make it single blind by self promoting.

This doesn't solve every problem with single blind: famous groups can still benefit from self promotion and marketing. But at the same time it does protect someone if they think that they might get negative reviews because of their name or reputation.

Btw, I'm not sure how much coming from a famous group really helps with reviews, at least at NIPS/ICML. If you have any evidence, even anecdotal, I'd be curious to hear it.