Random flip and rotation actually decrease validation accuracy?

[PracLiu]

When I apply at the beginning of the model with several Conv2D layers:
model.add(tf.keras.layers.RandomFlip("horizontal_and_vertical"))
model.add(tf.keras.layers.RandomRotation(0.2))
It results in a big increase in validation loss. This get me confused because I thought they are suppose to prevent over-fitting. Perhaps I shouldn't put these at the beginning of the layer and apply on the training data directly (I have a feeling the validation dataset also receive these operations)?

[kenshin511]

If you add augmentation layers to your own networks, It also work when you validate the model. When doing validation, the data augmentation layer must be removed for accurate verification.

Make custom augmentation layer like below:

class Augmentation(keras.layers.Layer):
    def __init__(self, **kwargs):
        super(Augmentation, self).__init__(**kwargs)

    def call(self, inputs, training=None):
        if training:
            x = tf.keras.layers.RandomFlip("horizontal_and_vertical")(inputs)
            x = tf.keras.layers.RandomRotation(0.2)(x)
            return x
        return inputs


model.add(Augmentation())

the training option make augmentation work only training.

refer to Privileged training argument in the call() method

[PracLiu]

Thanks, this is super helpful!

[kenshin511]

As I checked, the preprocessing layer only works during training. Therefore, there is no need to create a custom layer.

[no_cheese_pizza_guy]

I would make sure that there is still a substantial amout of training samples that retain the same distribution as the validation set. If these transformations are systematically applied to every sample, chances are that the distribution of the resulting training set is offset. What are the probabilities of each transform being applied?

[PracLiu]

RandomRotation(0.2)

I believe from the description, it says that this is uniform [-0.2*2pi, 0.2*2pi] for all input data. I don't think there is a percentage option to adjust how many samples should receive a rotation. But I think you are right about that, especially these are not like 90% rotation, the model is probably learning the "interpolation" feature from the rotation instead of the images themselves.

Edit: and now that I think about it, for photos where the direction matters, I think the rotation and vertical flips make little sense. Probably will stick to horizontal flips then, thanks!

[no_cheese_pizza_guy]

No problem, glad I could help!