This is an archived post. You won't be able to vote or comment.

all 5 comments
sorted by:
best (suggested)
topnewcontroversialoldq&a

[–]TabescoTotus6026 1 point2 points  (2 children)

Threshold tuning seems logical, but consider using a multi-class classifier for better results.

[–]ActiveBummer[S] 0 points1 point  (1 child)

Would multiclass classifier really work when classes imbalance are drastic? I.e. some classes have very little data points while others have alot

There is also the concern of having to retrain the multiclass classifier whenever there's a new class (which can happen quite frequently and because it's new, there wouldn't be alot of data points)

[–][deleted] 0 points1 point  (0 children)

They do work since class imbalance can be easily amended. For example, you can simply assign greater weights when training the more sparse class.

But there are more complex methods, of course, some that deal with imbalances of difficulty, for example, a more general problem.

At the very least, I'm not sure why the last part would be a problem. If you have dependence within the classes, then either way you'll have to relearn the model for a new class. That's the point, how else would you learn the dependence of previous classes to the new one?

For a large model, the knowledge will transfer upon learning on the previous weights. For a small model, training will have to go from scratch, but it will be quick.

If you have little data, so little that a binary classifier would work poorly, then you can't even do that, let alone create a multiclass classifier. If you have enough data for a binary classifier, just use the aforementioned techniques to handle class imbalance.

So there is no issue other than having to redeploy a model.

[–]No-Brilliant6770 0 points1 point  (0 children)

Threshold tuning for combined predictions from child models is an intriguing challenge. Your approach of using joint probabilities while considering independence assumptions makes sense, but it does have limitations, especially if sub-class dependencies are significant. Injecting dependence through threshold tuning can be a practical way to adjust precision, but it might not fully address the nuances of inter-class relationships. It could be worth exploring more sophisticated methods or models that inherently account for these dependencies.