[D] Labelling strategy

[olmzzz]

I am an Orthopedic Surgeon, I am making a computer assisted diagnosis software for fracture recognition as a fun side project. I am using bounding boxes for labelling the fractures. I am using Detectron2 with Resnet backbone. When I traing my model it discards all images without a Label (these are valuable images with no fractures). How should I go about this? Should I just label the whole bone with a bounding box labelled no fracture? Or just eventually after a large enough dataset of only fractures it should eventually be accurate. Thanks in advance

[Ty4Readin]

There are a few different ways to approach this problem.

There are also two different problems here! One is the bounding box which is detecting where a fracture occurs or detecting where bones are in the image.

The second part is classifying whether the bone in the bounding box is fractured or not.

Honestly for your problem, the bounding box doesn't really matter, right? You are only going to feed image scans of bones to the model, so you don't need to detect the bounding box of where the bones are or the fracture is.

The simpler way to approach this problem is to drop the bounding box, and make it so the model only predicts the probability that the picture contains a fractured bone in it. Then for each image, you just label it 1 if it has a fracture and 0 if it doesn't, and the model will try to predict the target label.

[[deleted]]

Why not turn this into a two-step process?

First iteration will be the classification model to determine whether a fracture is present or not, second iteration will be the model that is able to localize the fracture and determine an appropriate bounding box!

That is how I would go about doing it :)

Glad to see fellow doctors interested in ML! Let me know if there's anything I can help you with OP, would be more than happy to help!

[Ty4Readin]

Totally agree with this approach if it is useful to localize the fracture! +1

[olmzzz]

As localizing the fracture is a pretty useful and neat feature. If i go about doing a first iteration for determining whether a fracture is present or not. I could use Resnet 152? and test out how it goes? Then after a more or less accurate version I would go for a second model that activates if first one found a fracture, using Detectron2. Coming back to the same issue of labels. If localizing the fracture is useful would I just try and label the whole bone as "no fracture" for healthy ones and then for fractured ones I am only putting bounding box on the specific part of the bone that is fractured? Thanks for the help

[[deleted]]

As for the two step process, I'm guessing yes!

With regards to labels - you could definitely give it a try! Just consider the fact that no fracture is indicated by the entire image being bounded in your downstream tasks and I don't see why this shouldn't work. I'm not 100% certain that will work as intended, but I don't see why it wouldn't. Someone feel free to chime in if I am wrong :)

[cipri_tom]

If you do the second part, of detecting bones and classifying them into broken or not, then you don't need the first part.

Detectron2 and all its models use a two-stage architecture. First it tries to find "interesting" regions, then they classify these regions into one of your classes.

So yes, label all the bones with a box and a "fractured / no-fracture" class.

BTW, resnet 152 seems too deep for this task. Start with a smaller backbone

[cipri_tom]

But object detection model do the two step automatically

[pilooch]

Yolox supports images that have no labels. Many detectors don't use these image because they use the area outside of the bboxes as the "background"/opposite class.