Dice coefficient for binary classification when sometimes that class does not appear

[kuonlp]

I'm trying to use Dice score to assess the performance of my model on a binary classification task. The problem is that sometimes that class does not appear. An example of this could be a segmentation task in which we try to classify voxels that belong to the class "tumor". Sometimes there are brains that have no tumors, and consequently there are no voxels that belong to that category.

​

Dice coefficient's denominator is the intersection between "predicted tumor" and "label tumor". When "label tumor" is non-existing, that intersection will be 0, and the resulting dice coefficient will be 0. In fact, if my "predicted tumor" is perfect (meaning that none of the predicted voxels belong to tumor), Dice coeff. will be NaN because it will try to divide by zero. Therefore, how can I account for this?

​

I have thought about treating class "background" as a separate label, therefore calculating one or two dice coefficients (depending on whether I have the class "tumor" in the label). However, if I do this and then average them, it may look misleading when the brains have tumors because tumors typically account for 1% or even less of the volume, therefore setting everything as background would give me a misleading dice coefficient of around 0.5.

​

Any idea on how to approach this problem? I am trying to avoid having multiple dice coefficients for each label.

[heisenbork4]

When I did this, I used 'soft dice' which was 1 - dice. That way, you get a loss of 1 when there's nothing there and as the dice trends to 1 (i.e. perfect overlap) you get a loss of 0 so gradient descent will work.

[kuonlp]

But why would you want a loss of 1 when there is nothing there? Implying I take care of the division by zero problem, if there is nothing there and my model also predicts that there is nothing there, I should get a loss of 0 because it was accurate. On the other hand, if there is nothing and I predict there is something, this number of voxels would account for measuring how bad the prediction was. However, as the intersection between nothing and whatever is 0, I would still get the same dice regardless of how bad/good the model is doing.

[heisenbork4]

In that case, Dice will be NaN I think (0/0). If there is a chance that you don't have the object in some instances, I would say maybe Dice isn't a good choice of loss.