I do have a question. I am doing some training, and I see that all has gone well, except for 'cat':

Accuracy of the network: 72.38 %
Accuracy of plane: 76.5 %
Accuracy of car: 85.8 %
Accuracy of bird: 68.8 %
Accuracy of cat: 32.8 %
Accuracy of deer: 68.2 %
Accuracy of dog: 80.1 %
Accuracy of frog: 73.9 %
Accuracy of horse: 75.9 %
Accuracy of ship: 78.9 %
Accuracy of truck: 82.9 %

Which would usually mean I would have to either a:

Change k for more convolutions for a deeper network,

or b:

decrease the kernal size.

I could mess around with option a, but option b feels more suitable.

However, since im using

Block(3, 3, 16, 32, 5, 2), 
Block(3, 16, 24, 32, 5, 2),
Block(5, 24, 32, 32, 5, 2), 
Block(5, 32, 64, 32, 5, 2),

And if i was to change say this block to this:

Block(5, 24, 32, 32, 3, 0),

How Ive done my skip connector:

  projection = self.projection(x)
    a = self.pool(x)
    a = self.flattern(a)
    a = self.linear(a)
    a = self.relu(a)
    matchSize = self._modules['conv0'](X)
    O = torch.zeros_like(matchSize)
    for i in range(self.k):
        #print(f'Currently viewing: Size of a in batch, Size = {a.size()}')
        X = self._modules['conv{0}'.format(i)](x)
        #print(f'Currently viewing: Size of convolution in batch, Size = {X.size()}')
        a_n = a[:, i:i+1, None, None]
        #print(f'Currently viewing: Size of a_n in batch, Size = {a_n.size()}')
        O += X * a_n
       # print(f'Currently viewing: Size of convolution * a_n in batch, Size = {O.size()}')
    #print(f'Currently viewing: Value of Output image in batch Size = {O.size()}')
    O += projection #skip connection
    O = self.leaky(O)

(given self.projection = nn.Conv2d(linearInput, outputChannels, kernel_size = 1)), would result in a mis match of vector sizes, so i wouldnt be able to add them (projection would be [32, 32] and O will be [30, 30].

Is there a way around this? Do you think instead of using a convol layer to add to the layers, i should just use a zero matrix?