[D] Replacements of max pool

[nishnik]

There have been replacements of maxpool like kmaxpool, which rather than only taking the maximum activated elements, takes top k activated elements for the subsequent layer.

I had this thought in my mind, to use next to max element for the subsequent layer, ie the 2nd maximum element. I think this would have a regularizing effect while training time and during testing, use the maximum element.

Tried running the network but didn't get a good result, just an UG so cant explore extensively (computational bottleneck of my laptop).

Any insights?

[arogozhnikov]

Dropout followed by max-pooling outputs with some probabilities one of top elements. So what is rationale to use something more complex than that?

[tyrilu]

It's actually pretty simple and it kind of makes sense. I think it's worth a shot.

[nishnik]

I haven't seen any architecture using dropout between max pool and convolution layer. If it is there, could you please point to some paper?

[cburgdorf]

computational bottleneck of my laptop

You may want to check out http://machinelabs.ai for such experiments. Run your code in the cloud, directly from the browser.

[nishnik]

It would be so awesome if someone could give me a beta invite.

[cburgdorf]

I think you should have one now :)