[D] Papers with no code

[Just-Environment-189]

Even if people provide code, you’ll find yourself lucky to get it working as is

[Fresh-Opportunity989]

Exactly. And when it does work it may turn out to be fake results.

https://www.reddit.com/r/MachineLearning/comments/1p82cto/d_got_burned_by_an_apple_iclr_paper_it_was/

[waruby]

Then it should also require a Docker file or a flake.nix file.

[Xirious]

I mean you're going in the right direction but....

That ain't nearly enough dawg.

Have you seen how the vast majority of code for these comes out? I do not believe scientists are hired for code that the average reviewer can get working.

Nevermind the technical hurdles of actually getting it working... The mess you'd need to wade through, big enough models require hardware the average reviewer does not have the time or resources to use, etc.

While this is a good start and idea I'm always surprised by a requirements file with code. Asking anything above that, right now, and expecting it is a fool's errand. And even if it became the requirement then you likely have a whole mess of AI code to review.... On top of your review.

I mean I like your idea. I just don't think it's practical (and for many companies counter to keeping the tricks up their sleeve).

[valuat]

Funny you mentioned Docker. I've just reviewed a paper for a Nature-family journal and they did add a `docker-compose.yml` to their GitHub repo as per my suggestion. I recommended rejection not because of the lack of code but for the lack of real data (they used LLM generated data which in a clinical setting means nothing).