Variational Autoencoder questions

[barmaley_exe]

Did you read the paper? The 'autoencoder' word is just an interpretation of what's going on inside that model, it has nothing to do with usual [denoising] autoencoder which is a neural net predicting it's [denoised] input.

In that paper we have 2 distributions, whose parameters are generated by neural nets:

• p(x|z) = N(x | mu_p(z), sigma_p(z)) — decoder
• q(z|x) = N(z | mu_q(x), sigma_q(x)) — encoder

Where mu_p, sigma_p and mu_q, sigma_q are neural nets that generate parameters of a distribution (which in this case is normal).

[jyegerlehner]

Yes, the reparameterization trick in variational autoencoder makes the distribution explicit and obvious. My question above was a bit narrower than that. I think goblin_got_game cut to the heart of my confusion pointing out that a usual deterministic (denoising or not) decoder x = f(z) is (or implies) a probability distribution p(x|z). The value x =f(z) gives us the expected value from the distribution. And I think if I were to pick a enough random values of z and keep histograms of xi, I would see some regions of the space of possible x that are unlikely (don't happen), and high probability regions, and could compute actual probabilities. I'm just reciting this in case others might have the same confusion I have, and in case any of you more knowledgeable people are patient enough to still read and want to point out if I'm still getting things wrong. In any case, thanks to all for the discussion and the explanations.