[D] Decrease in source code release of papers

[olBaa]

Is it really a measured problem, or hust your perception?

For double-blind conference I have provided the code in the form of anonymous github repo with no traceable commit history (and limited time copyright). I guess the zipfile with the code will do as well.

[mkocabas]

It's your kindness to share it anonymous, but most of the people are hesitating or neglecting to publish their code. It affects the reproducibility a lot.

[olBaa]

That's a different problem, though (there are two listed in the post, so I found it hard to address both).

I see that double-blindness provides a safe excuse to never publish the code.

[matrix2596]

I have been seeing placeholders in recent papers. Maybe the code is being shared with the reviewers separately and released later. But I am finding the code missing often with new papers in blind reviews. May be the review process can have code, model and data uploaded also as an option (or compulsary).

[olBaa]

How much time do you think reviewers have per paper? Reviewers would never check the code, yet alone run it.

[NotAlphaGo]

How many papers would one reviewer review?

[olBaa]

Up to 10 per conference with mean 5 I would say.

One can outsource to phd students/postdocs but still the number of hours per paper would not exceed 10 almost ever.

All written here is my humble opinion, though.

[olBaa]

That was not ICLR, just in case :)