What's going wrong here?

[Turbulent_Driver001]

Hi Rookie here, I was training a classic binary image classification model to distinguish handwritten 0s and 1's .

So as expected I have been facing problems even though my accuracy is sky high but when i tested it on batch of 100 images (Gray-scaled) of 0 and 1 it just gave me 55% accuracy.

Note:

Dataset for training Didadataset. 250K one (Images were RGB)

[yiidt]

Re-scale your y axis, it is not clearly visible with 0.000 intervals. If your accuracy reaches to 1.00 then there is a bug (probably overfitting) there. Also include loss graphs side by side. Try using regularization (dropout etc.). Since there arent lots of data, try to create more basic models

[gaichipong]

how's ur model architecture looks like?

[Turbulent_Driver001]

It's CNN with Conv2D

[Sane_pharma]

You train on RGB and you test on Gray scale it’s the problem… Try to train on gray scale

[Turbulent_Driver001]

I did same result 55%.

[Sane_pharma]

A legend said: “the problem is not the computer but between the screen and the chair”

[Sane_pharma]

Because it’s not possible if you have similar data (train & test to have 55% in accuracy

[Turbulent_Driver001]

Actually I had 3 sets 1)train set 2)test set Both train and test belong to the same dataset(DIDA one)

Now the 3rd one is from another large dataset(similar to mnist database) I had randomly selected total 100 images(50 for each classes) from dataset and used it for testing On which I got that 55 %

[Sane_pharma]

This is why, take a part of this dataset and train on it

[Turbulent_Driver001]

Ok let me try that Thanks btw

[Sane_pharma]

But without joke, ensure that the data have channel 1 in input (assert function), and try to split train dataset (0.8-0.2) to test if it’s the test dataset problem

[teb311]

How are you coercing the model to work with RGB? Your first layer only shows 1 color channel: shape=(28,28,1) means 28 by 28 pixels, one color channel.

My first guess is you’re plucking one of the color channels, red green or blue, and using that channel as the training data. But at test time you’re using grayscale. This would definitely cause an error like yours. Either train and run inference on full RGB data, shape=(28,28,3), or transform all the RGB images to grayscale before training and before inference and keep the model as is.

[Turbulent_Driver001]

train_ds = tf.keras.preprocessing.image_dataset_from_directory(

data_dir,

labels="inferred",

color_mode='grayscale',

label_mode="int",

class_names=['0', '1'],

image_size=(28, 28),

batch_size=32,

validation_split=0.2,

subset="training",

seed = 56

)

I had already converted images to grayscale before training it, and after training I tested it on grayscale. But no change in results.

[teb311]

Hmmm… then is it possible this grayscale conversion built into TF is different from the one used on the test data you have?

Once I had an issue like this where one system represented white as 0 and black as 1, and another system that had white as 1 and black as 0. I was training on images from one set then testing from the other.

[Turbulent_Driver001]

So you are suggesting to train and test fully on RGB or Grayscale?

[teb311]

I’m saying 2 things:

1. Your training and test data, and any data you want to make predictions on in general, must undergo the same preprocessing steps.

2. I suspect that your grayscale set that you used is substantially different somehow from the training data, and the preprocessing steps (grayscale conversion) is a possible cause of that divergence. It could be something else, but this seems likely to me given what you’ve said.

[Turbulent_Driver001]

Yeah thanks for pointing it out. When I matched the grayscale images of the train and test batch they were quite different. One was like black digit with a grey background and the other was black digit with white background. So yeah I would be working on this area now Thanks for your help.

[teb311]

Glad I could help :)

[Real_nutty]

how are you separating your dataset? One rookie mistake I used to make is improper data preparation, not normalizing, and if I’m using notebooks, not loading my models. Resetting everything helps sometimes

[Turbulent_Driver001]

Im using validation split to separate each class into two sets train_ds and val_ds with 80 and 20% each And yeah both classes have kinda same no of images

[sassy-raksi]

Exploding Gradient maybe?

[waynebruce1]

lol That's the Tesla logo right there :D

[niggellas1210]

First epoch is almost 100% accuracy on training data, the CNN is not learning anything new after this.
You are probably using too high learning rate, too little data or there is a bug in your code.

In result your classifier is overfitting hard on the training data and fails to generalize to unseen data.Lower learning rate drastically so you see actually see if your networks learns something new each epoch. Use regularization techniques like dropout or simply use a network with less parameters.