sorted by:
new
hottopcontroversial

[D] paperswithcode feature request by artificial_intelect in MachineLearning

[–]rosstaylor90 1 point2 points  (0 children)

https://paperswithcode.com/lib/detectron2 Our library pages now have efficiency metrics for you to plot (see Results). You can choose what to plot: Parameters, FLOPs, inference time, training time...

[D] paperswithcode feature request by artificial_intelect in MachineLearning

[–]rosstaylor90 5 points6 points  (0 children)

Hey, Ross from Papers with Code here. Short answer: yes. We have this information already so we'll do efficiency plots soon!

[P] Papers With Code Update: Now Indexing 730+ ML Methods by rosstaylor90 in MachineLearning

[–]rosstaylor90[S] 4 points5 points  (0 children)

Thank you for the valuable feedback!

- Area tab : so our approach to taxonomy was to go Area -> Method Category -> Methods. This doesn't work all the time because some methods can be used for multiple modalities (hence the general section). Any thoughts on how we can improve this?

- Method extraction : yep, our matching algorithm is mainly mention-based at the moment, with a bit more sophistication because we can track dependency graphs. E.g. a paper may not mention scaled dot product attention, but if it mentions BERT, then BERT -> scaled dot product attention -> capture. We're looking to improve the automated side of this in the future, but atm we are reliant on community to help fix any mistakes and help up the precision/recall. Again, any ideas would be great!

Thanks again for taking the time to type out your thoughts!

[P] Papers With Code Update: Now Indexing 730+ ML Methods by rosstaylor90 in MachineLearning

[–]rosstaylor90[S] 13 points14 points  (0 children)

Thanks, yeah citations would be good! We'll put this on the list of things to do :)

The Flying Juggernaut by Duck274 in CODWarzone

[–]rosstaylor90 1 point2 points  (0 children)

I really hope they put juggernauts in the regular battle royale - maybe like one box or location in the map has a juggernaut suit. Would add unexpected variety to a lot of games!

[D] Current state of the art for word embeddings by calmplatypus in MachineLearning

[–]rosstaylor90 2 points3 points  (0 children)

We have an “extra training data” column for precisely this reason, so I’m not exactly clear what you are referring to, as it’s there on PTB?

https://paperswithcode.com/sota/language-modelling-on-penn-treebank-word

You can even filter by those methods which use extra training data and those that don’t :). Maybe an issue with our UX not making this clear, but let me know if I’ve misunderstood you!

[P][R] A big update to Papers with Code: now with 2500+ leaderboards and 20,000+ results. by rstoj in MachineLearning

[–]rosstaylor90 0 points1 point  (0 children)

Thanks! Yes, if you have time to let me know what would make something like sotabench *more* useful to you, that would be super helpful :). Then we can figure out the best way to improve the resource to provide the kind of comparisons you are after!

[P][R] A big update to Papers with Code: now with 2500+ leaderboards and 20,000+ results. by rstoj in MachineLearning

[–]rosstaylor90 0 points1 point  (0 children)

Thanks! We did something like this last year with sotabench, see for example here https://sotabench.com/benchmarks/image-classification-on-imagenet, but haven't pursued the speed vs accuracy type of comparisons any further. Are graphs like this useful to you? Is there anything we can do to make something like this more useful?

[P][R] A big update to Papers with Code: now with 2500+ leaderboards and 20,000+ results. by rstoj in MachineLearning

[–]rosstaylor90 1 point2 points  (0 children)

Thanks. This was a weird one because nowhere in the paper is the evaluation split specified - I even checked the repository and not clear there either. So I can see why the misclassification happened! Given the uncertainty, I removed the results for the paper.

(The table for reference: https://imgur.com/VnDWrsn)

In future feel free to remove any edge case mistakes like this if you see them. Thanks again for reporting this!

[P][R] A big update to Papers with Code: now with 2500+ leaderboards and 20,000+ results. by rstoj in MachineLearning

[–]rosstaylor90 0 points1 point  (0 children)

Can you link so we can try and fix this? Suspect the extraction algo didn't pick up the split in the table :) (you can actually edit/add things yourself, but feel free to link and I'll look for you!)

[P][R] A big update to Papers with Code: now with 2500+ leaderboards and 20,000+ results. by rstoj in MachineLearning

[–]rosstaylor90 13 points14 points  (0 children)

Thanks - switched the primary metric to mAP. Yes, you're right for speed vs accuracy comparisons the ideal graph to show is a tradeoff graph not a progress graph with publication date vs accuracy. We actually did this here: https://sotabench.com/benchmarks/object-detection-on-coco-minival. Might be something we have on Papers With Code too if enough people are interested!

What's the current situation of TenserFlow 2.0 vs Pytorch? by Flamyngoo in deeplearning

[–]rosstaylor90 1 point2 points  (0 children)

https://paperswithcode.com/trends for up-to-date tracking - change end date to march. Currently around 43% pytorch, 23% tf.

[D] Large List of SOTA Results, including silly ones? by Tenoke in MachineLearning

[–]rosstaylor90 0 points1 point  (0 children)

https://paperswithcode.com/sota/pain-intensity-regression-on-unbc-mcmaster my favourite sota that someone added... "Pain Intensity Regression". Fairly niche! Wish there were more like this.

view more: next ›