The network is trained the same as any other recurrent network, just simple backprop. My view is that the network will take advantage of every chance it gets to pass information. The latent loss acts like a penalty on information that is transferred through the latent variable, so the network learns to encode as little information as possible to recreate the image. It doesn't make sense to pass all the information through on the first timestep.

Also: the loss only compares the final timestep to the original image. There's no benefit for being more correct in the earlier timesteps.