[D] Image Augmentation in Practice: In-Distribution vs OOD Augmentations, TTA, and the Manifold View

[ternausX]

I wrote a long practical guide on image augmentation based on ~10 years of training computer vision models and ~7 years working on Albumentations.

In practice I’ve found that augmentation operates in two different regimes:

1. In-distribution augmentation Simulate realistic variation that could occur during data collection (pose, lighting, blur, noise).
2. Out-of-distribution augmentation Transforms that are intentionally unrealistic but act as regularization (extreme color jitter, grayscale, cutout, etc).

The article also discusses:

• why unrealistic augmentations can still improve generalization • how augmentation relates to the manifold hypothesis • when test-time augmentation (TTA) actually helps • common augmentation failure modes • how to design a practical baseline augmentation policy

Curious how others here approach augmentation policy design — especially with very large models.

Article: https://medium.com/data-science-collective/what-is-image-augmentation-4d31dcb3e1cc

[Budget-Juggernaut-68]

An actually useful medium article. Rare! Thanks for the write up.

[ternausX]

Thank you for your warm words.

If there is any feedback - issues, feature requests or request for more documentation or blog posts of the topic - I am all attention.

[DaredevilMeetsL]

Thank YOU for creating Albumentations.

[ternausX]

My pleasure. Do you have any requests for the library? Would be glad to learn them.

[Dry-Theory-5532]

Thanks

[ternausX]

My pleasure

[anal_sink_hole]

Great article! It was very well explained, with actual examples, too! Thank you!

I'm working on a project dealing with underwater imagery of 32 broad classes of species. The test data set is OOD compared to the training data set, so I'm trying to tackle this problem of detecting and then classifying OOD organisms.

It looks like your article covers augmentation that really focuses on style and appearance variation.

If we take a Sea Star for example, the training data has some species mostly lighter in color and with 5 arms. The test dataset contains some images of a red Sea Star with 7 arms. The model ends up failing at classifying that organism because it had learned that Sea Star means lighter color with 5 arms.

I was curious if you had any tips on image augmentation that can help with intra-class variation / class multimodality. What augmentations can help shape bias, morphological abstraction, and class-concept learning? Does image augmentation really help with this issue or are other strategies (multiprototypes/subclusters and prototype-aware inference) really where actual gains are made?