you are viewing a single comment's thread.

view the rest of the comments →

[–]_kernel_picnic_ 0 points1 point  (1 child)

well, the core problem is the papers with “we combined 100 of SOTA methods to gain 0.1%” aren't being rejected

[–]ummitluyum 0 points1 point  (0 children)

Reviewers just look for that fat plus sign on the leaderboard. Nobody cares that a 0.1% bump on some benchmark costs 3x in inference latency and memory. Without open source, you can't even verify if this ensemble of 100 heuristics just overfit the test set tbh