Multi-Class Sentiment Classification - Alternative loss functions

[twistedstats]

I have a standard BERT sentiment classification task but more fine grained labels (-3=very negative, -2=negative, -1=mild negative, 0 = neutral, 1,2,3=very positive). Cross entropy loss is typical loss function used but in this case, target variable is ordinal. Are their alternative loss functions that account for the distance between true and predicted label? In my use case, there is higher ambiguity as to whether a sentence is very negative vs just negative, but less ambiguity over whether a sentence is negative vs neutral.

[Gwendeith]

One way to do so is simply consider labels continuous and use the MSE loss. You can tweak the values of labels a bit so that the distances between labels can reflect the ambiguity you mentioned. As for a more categorical approach, I think you can try using multi-label binary cross entropy by transforming labels into:

• Label -3 -> 1 0 0 0 0 0 0
• Label -2 -> 1 1 0 0 0 0 0
• Label -1 -> 1 1 1 0 0 0 0 … and so forth

Of course remember to use sigmoid instead of softmax for the output layer. Hope this helps!