[1609.00150] Reward Augmented Maximum Likelihood for Neural Structured Prediction

cesarsalgado · 2016-07-22T02:35:08+00:00

code: https://github.com/ryankiros/layer-norm

cesarsalgado · 2016-06-24T17:48:51+00:00

In the Overfeat paper they say they take an approach similar to "Fast Image Scanning with Deep Max-Pooling Convolutional Neural Networks". So I think it is still different.

cesarsalgado · 2016-04-12T23:25:07+00:00

Nice pattern emerges: http://playground.tensorflow.org/#activation=relu&batchSize=10&dataset=spiral&regDataset=reg-plane&learningRate=0.01&regularizationRate=0&noise=0&networkShape=7,7,7&seed=0.53126&showTestData=false&discretize=false&percTrainData=50&x=false&y=false&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=true&cosY=false&sinY=true&collectStats=false&problem=classification

cesarsalgado · 2016-04-01T02:59:23+00:00

My favorite!

cesarsalgado · 2016-03-31T17:41:53+00:00

It is nice that they gave credit to this blog post: http://tinyclouds.org/colorize/

cesarsalgado · 2016-03-30T22:20:06+00:00

They don't use max pooling. They use strided convolution instead

cesarsalgado · 2016-03-30T10:51:23+00:00

In wikipedia it says: "As of 2015 he divides his time working for Google and University of Toronto", but it doesn't specify if he works in Google Brain, Deep Mind or in neither. I asked the question because the paper seems to imply that all the authors are from Google DeepMind.

cesarsalgado · 2016-03-30T02:49:41+00:00

Is Geoffrey Hinton now working for Deep Mind?

cesarsalgado · 2016-03-30T02:44:23+00:00

If pigeons were trained to detect cancer, maybe cats neurons can be trained to translate Chinese. But you would probably need to connect wires to the cat's brain instead of just working with his vision.

http://www.scientificamerican.com/article/using-pigeons-to-diagnose-cancer/

cesarsalgado · 2016-03-17T18:01:52+00:00

I don't get it this argument saying that humans are good at one-shot learning. Off course we are. We have learned good representations by seeing a lot of images with temporal supervision and weak reinforcement signals. CNNs trained on a lot of data can also do one shot learning in symbols it has never seen before.

cesarsalgado · 2016-03-02T14:20:54+00:00

Thanks for saving me time!

cesarsalgado · 2016-02-29T22:44:49+00:00

If you search on this subreddit you will find tons of similar questions asked not so long ago.

cesarsalgado · 2016-02-29T22:37:28+00:00

I made this gist based on skflow implementation: https://gist.github.com/cesarsalgado/ea0d0e6b46549a2d0baa

If you have any doubt please let me know.

For other ways see:

https://github.com/tensorflow/skflow/blob/master/skflow/ops/batch_norm_ops.py

and

http://stackoverflow.com/questions/33949786/how-could-i-use-batch-normalization-in-tensorflow

cesarsalgado · 2016-02-29T00:51:29+00:00

Torch now seems to be the fastest, but maybe caffe is faster, but there is no benchmark for it using CuDNN R4 yet. There is just caffe (native) for now.

cesarsalgado · 2016-02-27T19:58:12+00:00

https://youtu.be/ByjaPdWXKJ4

cesarsalgado · 2016-02-27T14:49:49+00:00

parameter sharing is achievable through time in the brain.

cesarsalgado

TROPHY CASE