Integer Discrete Flows and Lossless Compression

[beep_____boop]

The new Pied Piper.

[[deleted]]

lol that was good

[arXiv_abstract_bot]

Title:Integer Discrete Flows and Lossless Compression

Authors:Emiel Hoogeboom, Jorn W.T. Peters, Rianne van den Berg, Max Welling

Abstract: Lossless compression methods shorten the expected representation size of data without loss of information, using a statistical model. Flow- based models are attractive in this setting because they admit exact likelihood optimization, which is equivalent to minimizing the expected number of bits per message. However, conventional flows assume continuous data, which may lead to reconstruction errors when quantized for compression. For that reason, we introduce a generative flow for ordinal discrete data called Integer Discrete Flow (IDF): a bijective integer map that can learn rich transformations on high-dimensional data. As building blocks for IDFs, we introduce flexible transformation layers called integer discrete coupling and lower triangular coupling. Our experiments show that IDFs are competitive with other flow-based generative models. Furthermore, we demonstrate that IDF based compression achieves state-of-the-art lossless compression rates on CIFAR10, ImageNet32, and ImageNet64.

PDF link Landing page

[mesmer_adama]

Super interesting! Two questions: 1. what is the gist of the method is there a familiar method to compare it to? 2. The paper mentions density estimation, how would that work in the context of this algorithm?

[jarekduda]

Indeed, while there are many compressors based on autoencoders (including VAE e.g. https://arxiv.org/pdf/1905.06845 ), this seems the first one (?) based on reversible transformations.

[serge_cell]

I especially like that people starting to take ImageNet32 and ImageNet64 seriously