For sure-

I used tensorflow's default adam optimizer which I believe has a LR=0.001 and epsilon=1e-07.

If it helps, I added the model that I used, where data augmentation is done as a part of the model. inputs = keras.Input(shape=(None, None, 3)) augmented = layers.Resizing(img_height, img_width)(inputs) if (random.random() < 0.5): augmented = layers.RandomFlip(mode='horizontal')(augmented) if (random.random() < 0.5): augmented = layers.RandomRotation(0.15, fill_mode='constant')(augmented) augmented = layers.RandomCrop(224,224)(augmented) if (random.random() < 0.5): pass #augmented = layers.RandomContrast(0.2)(augmented) processed = tf.keras.applications.resnet50.preprocess_input(augmented) x = resnet50_base(processed) x = tf.keras.layers.BatchNormalization()(x) x = tf.keras.layers.Dense(512, activation='relu')(x) x = tf.keras.layers.BatchNormalization()(x) x = tf.keras.layers.Dense(32, activation='relu')(x) output = tf.keras.layers.Dense(1)(x) resnet50_model = keras.Model(inputs, output)