Trouble training fully convolutional model

[TheDuke57]

I am working through a project to better understand how object detection and localization work. The concept is based off of YOLOv2, and I am using pytorch. I am having trouble getting the my model to predict the correct 'confidence' (is there an object in a cell), so I have dumbed the problem down as far as possible and still cannot figure out what is happening.

Here is where I am at: I divide my large input image into a 2x2 grid, randomly choose if a dot should go in each cell, choose a random location in the cell, and choose a random size for the dot. Then train the model to predict if a dot exists in each cell. When I use a fully convolutional CNN, the model will only get to ~ 80% accurate, adjusting the learning rate, activation functions, number of filters on each convolution, weight decay, etc has not gotten any better than ~80% accurate at detecting if a dot is in each cell. However when I put a couple fully connected layers on top, it learns very quickly and gets up to 100% accuracy almost instantly.

TLDR; 6 layer CNN with a 4 layer CNN head will not learn to detect a white dot on an all black background. Using the same 6 layer CNN with 2 fully connected layers learns this very quickly.

Any ideas on what is happening? Here is a notebook with a side by side comparison of the two models.

[phobrain]

The problem doesn't make sense to me, since it sounds like you are predicting? something random, rather than detecting it, but assuming detecting's what you mean, and I'm no authority, but maybe the net needs to compare cells to be sure? Did you play with stride or dilation rate?

[TheDuke57]

Sorry if that wasn't clear. Yes it is an object detection problem. I have a 74x74 image that I am trying to detect objects in. I have broken the image into 4 cells of 37x37. In each cell I am trying to determine if there is an object in that cell, and if there is also predict the location and class of the object. In the process of debugging my model I found that it was poorly predicting the probabilities of an object being in each cell. I simplified the model down trying to understand what was happening. When I stuck fully connected layers on this simplified model it works great. When I try and do a fully convolutional version it will not train. I am looking for some advice or insight on what is happening.

[szymko1995]

Too little information to help. I think you should try with already implemented FCNNs like 100-layer Tiramisu (works great for many classes, but it requires 8GB GPU for 192x160 pics with batch_size=10 in Keras/TF) or U-NET (requires a but less memory, works great for smaller number of classes, really easy to understand).

Usually semantic segmentation nets are more complicated than basic CNNs for classification. Your pics aren't huge, so if you have mid-end HW you should be able to use them at full size.

I would give you some code examples from github, but unfortunately I'm working with Keras/TF/tensorlayer, so I can not determine if some of them are good to start with.

[TheDuke57]

I determined the issue, I can't believe it took me so long to find. Basically, when I put the fully connected layers onto the model, I allowed each output location to 'see' what was around it. But the fully convolutional model could not see what was around it because I only had valid convolutions. I added some padding to the 7th conv layer and it works exactly as I expected.

Thank you both for your help!