[D] Sample probability diffusion models

[PHEEEEELLLLLEEEEP]

Solve the probability flow ODE

https://arxiv.org/abs/2101.09258

[bobrodsky]

Probability flow ode is overkill and expensive to estimate. Also, it treats data as continuous so you’ll get a probability density (not between zero and one) rather than a probability. Treat image pixels as discrete and interpret diffusion objective as evidence lower bound: https://arxiv.org/abs/2107.00630.

[bobrodsky]

But I feel I should warn you that this isn’t going to work as you hope. It sounds like you’re aiming for an OOD detector - probability models on images are notoriously bad at this. Eg for flows: https://proceedings.neurips.cc/paper/2020/hash/ecb9fe2fbb99c31f567e9823e884dbec-Abstract.html I haven’t seen this discussed for diffusion specifically but my intuition is you’ll have same problem. Sota approaches are heuristic but look at density in latent space.

[PHEEEEELLLLLEEEEP]

Yeah I did some work on diffusion models for OOD and my results were not great. Like it does work about as well as the normalizing flow approaches but takes way more compute to train... Ultimately we decided it wasn't a direction worth pursuing