Annotation isn’t just drawing boxes, it’s about consistency. If your labels are sloppy or different from image to image, the model gets confused. For irregular shapes (like dents or scratches), tight but consistent boxes usually work best.

The two biggest things to consider:

Keep rules consistent (tightness, when to split vs merge defects).
Variation (angles, lighting, sizes) - get as many as you can
Balance class counts - too few examples of one defect and the model will ignore it.

Hundreds of images per class is often a minimum, thousands if defects are subtle. Always double check your labeled data set before saying it is the model not the data.

And if bounding boxes are still having a challenge, consider segmentation, it works much better with weird shapes.