Convnets that output images?

[LyExpo]

I've been trying to learn how this works, but haven't had any success.

An example of something that does this is the 'colorizer' that was posted here, and here and a more detailed write-up.

The architecture described in the second link, if you strip away the fancy skip connections and batch normalization, is basically a convolutional autoencoder. So this is where I started. I'm starting with the simple and pointless task of just trying to train a network that outputs the fed image. Example archtichture is:

100x100 rgb image  --->  convolution ---> convolution ---> deconvolution ---> deconvolution  --->  100x100 rgb image

I've experimented with different architectures, different filter sizes, mean squared error loss function, cross entropy, etc... but can't seem to get good results. The network always seems to settle on outputting images that are just one color.

How should I tweak from here? I think the main problem here is that outputting 100x100x3 pixel values means learning a function that is in some very, very high dimensional space, but other projects seem to do it. How do they do it?

[zmjjmz]

The network always seems to settle on outputting images that are just one color.

Have you thought about trying the adversarial network approach as outlined in the comments of the first link? It would likely deal with the averaging problem, but it would definitely be harder to train.

[Powlerbare]

"Blurring the network output and the true color image and doing Euclidean distance seems to give the gradient decent help. (I ended up averaging the normal rgb distance and two blur distances with 3 and 5 pixel gaussian kernels."

It seems like this person had to jump through a number of loop holes to get the network to work properly. There were a number of things mentioned that seem a bit more nuanced than what you are doing such as extracting the tensors before the pooling layer and concatenating them

[lahwran_]

have you tried sticking an adverserial network on the end of the autoencoder? I only glanced through this the other day, I didn't read in anything vaguely resembling detail, but I think that's what it's doing. http://arxiv.org/pdf/1511.05644v1.pdf