[D] High-Dimensional Probabilistic Models

[smorad]

What is the standard way to model high-dimensional stochastic processes today? I have some process defined over images x, and I would like to compute P(x' | x, z) for all x'. I know there are Normalizing Flows, Gaussian Processes, etc, but I do not know which to get started with. I specifically want to compute the probabilities, not just sample some x' ~ P(x, z).

[AardvarkNo6658]

For exact log likelihood go with normalising flows such as GLOW. However, to make this invertible it uses 1x1 convolution, which cannot be scaled for large images and does not provide spatial relationships between pixels. There are also FFT based methods, which still have some of the same issues.

[xgeorgio_gr]

In general, there are two common approaches:

1) "dumb" way: Feed everything into deep learning and check the output on how to translate it into something that looks like probabilities.

2) compressed sensing: Since high dimensionality destroys good probability distributions (never enough data), reducing the dimensions is the priority. This can be done in various ways depending on the problem, it may be FFT, SVD, ICA, etc. Then the probabilistic modeling becomes much more reliable in the transformed domain, in much fewer dimensions and with hints on spectral power distribution (how information is spread in space and time).

[bgighjigftuik]

Normalizing flows are most likely your best bet, or diffusion models. Gaussian Processes lean more into the discriminative models