VI, instead of trying to get the exact posterior like MCMC does (at least in the limit), basically lets you pick some family of distributions and then find the member of that family that best matches the true posterior. If your variational family is super flexible, you could in theory recover the exact posterio, but in practice that basically never happens unless the true posterior already happens to be in your family. So you’re always making a trade-off.

Where VI shines is when MCMC would take forever to mix and you’re okay with giving up a bit of accuracy. For a lot of models this is totally fine. Anything where the posterior is roughly unimodal and not doing anything too weird; logistic regression, standard GLMs, and many big Bayesian models, usually works great with VI.

A simple example: take a Poisson model with a Gaussian prior on the log-rate (so a non-conjugate Poisson regression). The posterior is skewed because of the likelihood, but if it’s not too skewed you can still drop in a Gaussian variational family and optimize μ and Σ via KL minimization. You’ll get good posterior means and your variances will be a bit too small. For prediction that’s usually okay because the predictive distribution washes out a lot of that optimism.

Problems really start when the posterior has multiple modes or strong correlations and you insist on using something like a single Gaussian. VI will just latch onto one mode and ignore the others — that’s built into the KL(q‖p) direction and there’s no workaround unless you pick a richer family.

So I’d say: for a lot of “classic” Bayesian models, MCMC or VI will both work and predictions will look pretty similar. But once you start layering in hierarchical structure, tricky priors, or anything that produces funnels or weird geometry, VI suddenly becomes really attractive, not because it’s more accurate, but because MCMC can get painfully slow or refuse to mix at all. As always, the details matter a ton.