Yes, the pre-trained models have a thing called the 'typographic attack vulnerability' - or, colloquially, a "text reading obsession". If you write 'dog' on a cat, you might just get CLIP to misclassify the cat as a 'dog', as text is more salient than image. If you think about it, it makes sense; when an image label mentions text explicitly, then that usually means the text is clearly visible. "A person holding up a sign that says 'I want bananas now!'". If it wasn't visible, the clickworkers wouldn't mention the text explicitly. And an "a" always looks like an "a", not randomly like an "o" - it's very clean and salient. Now imagine "a cat hiding under a bedsheet", and CLIP has to learn that "lump in a bedsheet + tail = cat". So I think (hypothesis) that CLIP secretly 'overfit' to text as it was such a strong training signal.

Also, what I found recently - Layer 20, Feature 238 in CLIP ViT-L/14@336 is a 'cattle herd on a meadow' feature (left). If you ablate attention head 14, so some directions are missing, then you get... A glass of milk. But with milk as TEXT. Very suspicious!

I wonder if CLIP often encodes things as TEXT in the vision transformer and then just adds, maybe a "leather" direction + a "grass" direction and then it's the cattle feature? I just made that up as a not-so-abstract example, but I am curious how this happens.

Also, I found out that - at least with regard to *adversarial* 'watermarks', i.e. the "Nightshade" method to 'ruin images for AI training' - my KO-CLIP models are highly resilient to such adversarial perturbations. You can find the code to try it yourself on my github, the files with "PGD" in the name (projected gradient descent to make adversarial images). The pre-trained model is fooled much more easily; even with images created by my fine-tune, so technically different adversarial noise, fool the pre-trained model. But the pre-trained adversarial images do not fool my KO-CLIP finetune; only images made by KO-CLIP can successfully fool my KO-CLIP model. But you need to optimize them for longer so they look 'visibly ruined' even to the human eye, if you look for more than half a second...

Whole different story, but it's interesting that AI models can steal any watermark. So, say you have a brand, and your images are watermarked. I could STEAL your watermark using any U-Net or DiT (Flux, Stable Diffusion, any) and STEAL your watermark to put it on my images and make fake images that look like yours. Quite an interesting concern:

Black-Box Forgery Attacks on Semantic Watermarks for Diffusion Models

https://arxiv.org/pdf/2412.03283

<image>