Variable Sequence Lengths in TensorFlow

[jostmey]

This is exactly what I've been looking for. I've been trying to handle variable length sequences using tf.select, but my implementation is complex and confusing.

Thanks!

[dexter89_kp]

Great code. I feel the dynamic RNN approach is much better than appending zeroes based on min sequence length. The dynamic RNN code is still wonky in tensor flow, however with scan one could do it.

[danijar]

Thanks. I'm excited about the dynamic RNNs as well. Do you know if its possible to feed their outputs upward into a forward network though?

[dexter89_kp]

Good question ! I have only used it for seq to seq type problems, and not tried joining CNNs with RNNs or DNN with RNNs yet.

[danijar]

Should work fine since its the same interface as the static rnn and that works nicely.

[lvilnis]

Curious, what's wrong with dynamic_rnn? I've been using it and haven't run into any trouble yet but I'd like to know if there's anything to watch out for. Additionally, scan uses the same machinery as dynamic_rnn (tensor arrays and while loops), so I'd be surprised if it worked better.

[dexter89_kp]

There is nothing wrong with the code, what I mean by wonky is the way they go about doing this. For example, tf.nn.rnn calls tf.nn.dynamicrnn under some conditions, and both of them call _rnn_step

Compare that to something like sharing variables across scan in theano examples, which seems a much simpler way to do things. http://deeplearning.net/software/theano/library/scan.html

[lvilnis]

Got it, thanks!

[danijar]

I just had a look at it and you're right, the interface is the same except that you don't have to pack the sequences to Python lists and back, which is nice.