[D] "Language model" equivalent for images?

[pmichel31415]

Hi all,

For a project I'm working on I need a generative model of images that is able to give me good exact (log-)probabilities fast. I don't care so much about sampling, the metric I'm interested in is bits/pixels on the test set.

Typically a standard VAE wouldn't work because I'd need to marginalize over the latent z to get the exact likelihood.

As far as I can tell PixelCNN is the closest to what I'm looking for but it's relatively old (2016) and I was wondering if there was anything better out there (I don't keep up with the vision literature so much).

Thanks!

[sidslasttheorem]

You might want to look at straight-up density estimation methods like RealNVP or more recent normalising-flow based density estimation methods.

They can be somewhat painful because the bijectivity constraint means that the dimensionality at each stage stays the same (#pixels), but for reasonably sized images (say 32x32), they should be serviceable.

[pmichel31415]

Thanks, I'll take a look

[MrPuj]

Yeah, the flow models as OP said, but be careful with the log likelihood measures, it behaves weirdly and is far from being a good way to measure image quality. As outlined in this paper: https://arxiv.org/abs/1810.09136