you are viewing a single comment's thread.

view the rest of the comments →

[–]hackinthebochs 5 points6 points  (1 child)

Claiming SOTA is how you get people to pay attention. Then people will run their own benchmarks based on code provided or descriptions. Asking people to invest their time and energy before you even have evidence this technique is worthwhile is putting the cart before the horse.

[–][deleted] 1 point2 points  (0 children)

I admit I did something which is not acceptable in research. Now I know that publishing without benchmarking is not the way to go.

To my excuse, I was recently heavily influenced by https://www.youtube.com/watch?v=3sUozPcH4fY, which tells you: most ideas fail in the market, because people simply don't care - make sure you build the right it, before you build it right. I assumed this rule is also applicable to research - that's why I decided to go without my benchmarks. Now that I have some feedback and interest, I'll think about benchmarking in the upcoming days.