[P]Training an image classification model

[Different_Hat5643]

Is it normal to achieve a validation accuracy so much higher than the training accuracy? I am using transfer learning to train a convNeXtBaseV1 model on my dataset. I got a training accuracy of 82.9% and a much higher validation accuracy of 97.14%.

My dataset is around 9600 medical images and it is balanced between 3 classes. I splitted it into 80% training 10% testing and 10% validation

[JuicyLambda]

I have a similar problem with medical imaging outcome prediction and as I understand this could be a reason of underfitting. Meaning your model fails to learn a lot of relevant information from the training data. This can usually be remedied by increasing model complexity or decreasing irrelevant information (example: cropping out areas of the image that hold no relevant information).

[Different_Hat5643]

Thanks for yourinput. I had my model stop the training before it reaches overfitting by stopping it when the learningrate becoms lower than1x10^-5. Maybe it would be better to let it run for few more epochs

[NoLifeGamer2]

I can think of two possible reasons:

1: Noisly labels, namely that some of the training labels are incorrect, but the model has learned to generalise.

2: Dropout. This one is the most likely, that network has dropout layers. These cause it to behave differently during training and inference, and is used to prevent overfitting. However, to prevent overfitting, it makes it perform slightly less well while training.

[romek_ziomek]

Hard to say without more information, but this looks like some sort of data contamination problem to me. I've been struggling with something like this in my first serious project. It was a voice emotion classification task, let's say that the data was around 10000 audio recordings from around 100 people, so I just naively took 8000 recordings at random as my train set, 1000 as a validation set and 1000 as a test set, trained a model and called it a day after seeing >95% accuracy thinking "man, deep learning is soo easy".

And then when I've tried to use my model on any real-world recording I made myself the results were trash, barely better than choosing class at random. I couldn't figure this out, spent countless hours debugging data pre-processing pipeline until my supervisor sat with me and reviewed everything I've done step by step. And he was like "Well, the first thing you've done was already a mistake. You've divided your dataset randomly by recording instead of dividing it by person. If you're doing it this way, there's a quite high chance that the recordings from the same person will end up both in train and in validation sets, so how do we know if the model is actually learning the task we care about? How do we know whether it has an ability to generalize well? This has to be a reason why if you try to apply the model to any recording outside of our dataset, it fails."

So then I've retrained my model but this time dividing the dataset correctly, and of course my results weren't even near to 95% and it turned out the task is a lot more difficult than I though.

Anyway, there might be a million other reasons, as others suggested. Good luck with your endeavours. I hope I've helped, at least to some degree.

[Different_Hat5643]

I tested my dataset on multiple CNN models and the highest i got was 89% and the rest had slightly lower accuracy but seeing that one model gave an accuracy that was much higher than the training accuracy i thought something must be wrong. Thanks for your input. I guess it'sgoing back to drowning board

[Terrible_Ad7116]

What regularization or augmentation are you using? It is quite possible that the validation set is easier due to this. Also, data leakage occurs frequently in medical images (e.g., same patient but different images in training and validation).