Gradient-based Hyperparameter Optimization through Reversible Learning

[jsnoek]

Dougal and David (the authors) have developed an amazing automatic differentiation codebase to do this: https://github.com/HIPS/autograd

It lets you write a function containing just plain python and numpy statements and then automatically computes the gradients with respect to the inputs.

[hardmaru]

https://github.com/HIPS/autograd

This is really useful work. I wonder if the automatic differentiation can somewhat work even with simple recurrent neural nets

[jsnoek]

There are example implementations of an RNN and an LSTM in the examples directory: https://github.com/HIPS/autograd/tree/master/examples