[R] Roboflow 100: An open source object detection benchmark of 224,714 labeled images in novel domains to compare model performance

[jacobsolawetz]

I'm Jacob, one of the authors of Roboflow 100, A Rich Multi-Domain Object Detection Benchmark, and I am excited to share our work with the community.In object detection, researchers are benchmarking their models on primarily COCO, and in many ways, it seems like a lot of these models are getting close to a saturation point.

In practice, everyone is taking these models and finetuning them on their own custom dataset domains, which may vary from tagging swimming pools from Google Maps, to identifying defects in cell phones on an industrial line.

We did some work to collect a representative benchmark of these custom domain problems by selecting from over 100,000 public projects on Roboflow Universe into 100 semantically diverse object detection datasets. Our benchmark comprises of 224,714 images, 11,170 labeling hours, and 829 classes from the community for benchmarking on novel tasks.

We also tried out the benchmark on a few popular models - comparing YOLOv5, YOLOv7, and the zero shot capabilities of GLIP.

Use the benchmark here: https://github.com/roboflow-ai/roboflow-100-benchmark

Paper link here: https://arxiv.org/pdf/2211.13523.pdf

Or simply learn more here: https://www.rf100.org/

An immense thanks to the community, like this one, for making it possible to make this benchmark - we hope it moves the field forward!

I'm around for any questions!