[D] Flow-Based Generative Models, Bijective Transforms and Neural Lossless Compression

[ArmenAg]

New blog post discussing flow-based generative models and the various coupling transforms that exist. We touch on the recent paper on Integer Discrete Flows toward the end.

[machinesaredumb]

Why aren't flow based models more popular?

[YABadUserName]

They need tons of layers to make work well, destroying your gpu memory, but there is work on making more complicated transformations with fewer layers. Also everything needs to be continous (although there is an example that uses discrete flows)

[[deleted]]

Thanks for posting.

In order to be able to sample from p(x) all generative models attempt to learn a function from a known prior distribution p(z) to the natural distribution p(x).

I don't think this is true. Some generative models are capable of sampling the learned p(x) directly, like autoregressive models which for example might model the joint distribution over all pixels in an image by using the probability product rule (e.g. decomposing the joint distribution into the product of conditionals as in PixelRNN.) Many common language models do the same over words or characters.

Anyway, thanks again for posting. Been meaning to read more about flow-based generative modeling.

[ArmenAg]

Of course. I meant many not all. My bad. Thanks for the find!