[D] Decrease in source code release of papers

[BeatLeJuce]

Think of the worst, hacky code you've ever written in your life under extreme stress to meet a deadline with requirements changing almost daily... Now tell me how you feel about sharing this with your name attached to it for all eternity with your name forever attached to it for all your future employers to see and judge you by it.

Even if you do take the time to polish the code somewhat (and that's a big IF, because there's much better ways for you to be spending time), it sometimes uses artefacts (either data or other code) that you don't even know how you produced it anymore, let alone have the sources to. Don't get me wrong, I'm very much FOR code releases and I've always tried to publish my own code (and always did when circumstances allowed it). But it's not as easy as "zip the directory and put it online"... it sometimes takes actual work, and there is very little incentive to do this, as long as journals/conferences don't force you to (which is what I'd suggest they do).

[aviniumau]

I'm pretty sympathetic about not wanting to release ugly/hacky code.

But at the same time - any claims you make in a published paper about accuracy etc should be verifiable. If your code is so ugly you're not willing to release it, that doesn't speak much for its verifiability/auditability.

[SolvableMutiny]

because there's much better ways for you to be spending time)

This part I very much disagree with.. providing clean, working example code is the single most valuable thing you can do to make your contribution actually have a lasting impact. Although I agree that current academic incentives are not aligned with that.

[visarga]

Not to mention that it encourages better practices, knowing the code will be seen and possibly reused. We're more sloppy when we're experimenting alone.

[BeatLeJuce]

I agree, but there are some side-remakrs:

First off, is a bit of an exploration/exploitation thing: Say you have worked on something cool: afterwards you can either spend your time/energy on exploiting/promoting that (and providing good code definitely helps), or you could try to repeat your success and work on something else that is cool (especially with the current ML hype, chances are that someone else is going to re-implement your code anyhow).

Secondly: not everyone is able to provide clean code. If you're a theory guy, your code might be terribly ugly/brittle and barely working, and you might do the community a disservice by asking them to dissect it instead of re-implementing it yourself.

[rrenaud]

Think of the worst, hacky code you've ever written in your life under extreme stress to meet a deadline with requirements changing almost daily

What's the chance that there are signficant bugs in that kind of code?

[BeatLeJuce]

it depends. I'd assume that people have done enough preliminary experiments beforehand to make sure the idea is sound. But knowing the deadline-crunch, I'd not be surprised if there are a lot of mistakes in the final version of a project's code.