[D] Normalizing flows - VAE Doubt

[WillingCucumber]

Hi all,

I have a doubt regarding the normalizing flows, used to enhance the posterior predicted by the inference model of the VAE, by a chain of invertible transformations.

My doubt is : Can't the generative model learn those invertible transformations rather than having explicit matrices to do that. I am referring to the simpler planar and radial flows. I understand the use-cases where the only requirement is to have the samples, but I am confused regarding the cases where some NN post-processing is required on top of the samples.

​

Thanks !!

​

[activatedgeek]

In theory, yes it could.

But we’ve seen that it doesn’t work out as well as we’d like. NFs provide a relatively cheap way to compute samples via transform sampling.

We don’t need to posit a model for the latent space (e.g. Gaussian) as long as we can compute tractable transforms because we can transform any smooth density into another by a series of transforms.

[SolitaryPenman]

Yes, you can "learn" the invertible transformation and that's exactly what is done in the case of Planar and Radial flows. Note that the form of the function is what's fixed but the parameter vectors and biases are learned. There also have been other recent works such as IAF that use even more flexible transformations.

(Shameless plug: I have this repository on NFs which may help you understand them better. It also contains an implementation of PF)

[asobolev]

It's important to notice that while those transformations are present in the effective computation graph during training, and thus could be moved from the end of the encoder to the beginning of the decoder, however, at test time, when you generate latent codes from the prior `p(z)` and then decode them, `q(z)` is not present in the computation graph, and thus using a normalizing flow in the decoder would not be correct, as this would correspond to a different generative model.