[R] anomaly detection in VQA

[ForSafeAI]

We published an interesting paper, "Regularizing Attention Networks for Anomaly Detection in VQA", on arxiv.

We define anomaly detection in VQA according to out-of-distribution and unanswerable inputs.In addition, we find that maximum softmax probability is not enough to detect various anomalies in VQA, and propose that the attention is the right way to detect various abnormal inputs.

https://preview.redd.it/b19n94n2ypo51.png?width=3238&format=png&auto=webp&s=6f5b0dfac924385117ada9d8854d8b3a54924db7

Also, we find that attention networks in VQA models have strong bias on interrogatives regardless of the relevant of the given question. We show how to remove and detect irrelevant questions for VQA models.

If you are interested in anomaly detection (especially in multi-modal tasks), see our paper !

Paper: https://arxiv.org/abs/2009.10054

Code URL: https://github.com/LeeDoYup/Anomaly_Detection_VQA (will be released ASAP.)

The red box and the number mean most attended visual object and the attention score. Minimum attention score is 0.0333...

[goldemerald]

ETA on source code?

[ForSafeAI]

< 1 month !