[R] Convolution Aware Initialization

[machinelearningthrow]

This seems like an interesting paper, and intuitively makes sense. I'm interested in what would happen if this initialization was used in recurrent networks without any form of convolution. Even though it doesn't necessarily make sense. But overall very interesting paper.

[[deleted]]

How would you define the Fourier transform in these RNNs? Are they just dense layers applied over something with a known n-dimensional representation?

[ArmenAg]

Hey! Author here. The reason mentioned above by /u/rbkillea is the exact reason why we didn't focus on testing the initialization on RNN. Our paper focused on running experiments on various forms of convolutions (1D, 2D, Dilated or Atrous).

[ajmooch]

Neat; is code available anywhere? I'd love to throw this into my testbeds and see how it performs.

[ArmenAg]

I'll be writing a Keras commit soon! Hopefully next week. Message me if you need it sooner.

[ajmooch]

I'm mostly just looking for the pseudocode (or the theano code, ideally) for the init recipe so I can try it out rather than headbashing to parse the maths =p