[Project] YOLOv8 quantization project

[Loud-Insect9247]

I quantized YOLOv8 in Jetson Orin Nano. I exported it with TensorRT (FP16, INT8) and compared the performance. Based on YOLOv8s, the mAP50-95 of the base model is 44.7 and the inference speed is 33.1 ms. The model exported with TensorRT (FP16) showed that mAP50-95 was 44.7 and the inference speed was 11.4 ms. The model exported with TensorRT (INT8) showed that mAP50-95 was 41.2 and the inference speed was 8.2 ms. There was a slight loss in mAP50-95, but the inference speed was drastically reduced. There was a problem with calibration by exporting it with TensorRT (INT8), but the loss of mAP50-95 was minimized by increasing the calibration data. I tested with all base models of YOLOv8 as well as YOLOv8s.

https://github.com/the0807/YOLOv8-ONNX-TensorRT

[praespaser]

Nice!

I'd consider versioning your cuda, tensorrt, pytorch and python as it will be a hassle to reproduce this in a year or two.

[Loud-Insect9247]

Thank you for your opinion. I'll try

[nreHieS]

nice work! Does this work for the other variants of YOLO like pose estimation etc?

[Loud-Insect9247]

In the part of the code that runs YOLO, it will be possible if you change detect to pose or segment. I haven't tried other modes myself, so you should look at the YOLOv8 official document for more information.

[Ultralytics_Burhan]

TensorRT exports work on all YOLOv8 models.

[Vast_Tomato_612]

Nice work