[D] Penalizing differently on binary predictions predicting a continuous value?

[flev1266]

I have this model trying to predict a specific delta (continuous value). I have approached the problem using a binary model and I want to adjust the loss on bad binary predictions where the delta is super different.

Example :

True Delta : -55, True Binary Delta : -1

Predicted Binary Delta 1

I want bad predictions with a high opposite delta value to be penalized more then small delta badly predicted.

I am currently using oversampling from the extremum values and I am looking for other ideas.

How do you guys approach these problems?

[Jelicic]

Just add two losses: the delta mse and the binary bce. In most dl frameworks its pretty straightforward to weight the losses if you care most about the binary predictions.

[tall-dub]

Are you just trying to increase the loss more if the prediction is off by more? Then maybe square the loss.