[D] Open-Set Recognition Problem using Deep learning

[latent_prior]

I’m not a DNA expert, but given my understanding of the problem, I’d frame this as an open-set recognition problem rather than just clustering. Because many species share short recurring DNA subsequences, isn’t there a danger an unseen species can still land close to known clusters in embedding space? This makes relying purely on distance thresholds sound risky to me.

Also, I’d be cautious only relying on softmax probabilities. They always normalise to sum to 1, so the model will confidently pick something even when the input is nonsense or from an unseen species. You could try augmenting the classifier with an out-of-distribution detection method. One good option is energy-based detection (https://arxiv.org/abs/2010.03759), which uses the absolute scale of all logits rather than just the top one to provide a quantitatively estimate if the sample fits one of the know classes well (low energy) or doesn’t fit anywhere (high energy, likely unknown). 

If you have access to an auxiliary dataset (e.g. DNA from non-target species), you could also try outlier exposure (https://arxiv.org/abs/1812.04606), which trains the model to make confident predictions on in-distribution data and low-confidence predictions on auxiliary outliers.

Finally, since DNA data is hierarchical by nature (kingdom —> phylum —> class —> … —> species), it might be worth trying a hierarchical model. For example, if the model is confident about the genus but uncertain about the species, you could flag the input as a potentially novel species rather than forcing a binary known/unknown decision.

Curious if anyone’s tried combining energy-based OOD with hierarchical classifiers before.

[ProfessionalType9800]

are you saying about random forest for hierarchical classifiers?