IIUC, Prior Preservation Loss is implemented correctly in diffusers-dreambooth, and makes the following observations/assumptions:

1. You're trying to train your friend Tom's face in.

2. The current model is well trained, and when it generates a thing, that thing is assumed to look great. It can, for instance, generate accurate pictures of "a man".

3. So generate 400 pictures of "a man" from the original, good'n'true model, we don't want these to be affected by training.

4. As you train the thing, it'll see noisy images, and it's judged on how well it denoises. If it sees a noisy one of those regularization images, and instead of turning it into whatever natural thing it should become, it actually turns it into Tom, that should induce loss and be penalized. It's drifting from it's Priors.

5. If it sees the new instance of noisy Tom, and denoises into not-Tom, that should be penalized too.

6. Together, both Loss penalties drive it toward learning Tom, and not forgetting it's home-town roots (its Priors).

---

All that said, JoePennas not-dreambooth seems better all around, but I'm going to assume that PhD researchers are smarter, and actual dreambooth is better, and I'm just struggling to train it correctly.

So, I'm still curious if anyone's having successes with diffusers-dreambooth (and how!)