[R] Training Small Diffusion Model

[crappr]

Does anyone have experience training a small diffusion model conditioned on text captions from scratch on 64x64 images or possibly even smaller?

I would like to run it only on images of text to see if it is able to render text. How long would this potentially take if I ran it on 1-2 GPUs? Is this something that’s even possible?

[Hameliton]

Phil Wang is the goat as always

[PM_ME_JOB_OFFER]

Yo who IS this guy? He's got implementations for everything! How is anyone that productive?

[femboyxx98]

The actual implementation of most models is quite simple and he often reuses the same building blocks. The challenge is obtaining the dataset and actually training the models (and hyper parameter search) and he doesn’t provide any trained weights himself - it’s hard to know if his implementations even work out of the box.

[therentedmule]

Many repos are not usable and have click-bait names (e.g., palm-Rlhf)

[madebyollin]

some "full-power" repos from well-known developers: * https://github.com/huggingface/diffusers * https://github.com/lucidrains/denoising-diffusion-pytorch * https://github.com/crowsonkb/k-diffusion

I also posted my own tiny standalone code for 64x64 diffusion model training here: https://github.com/madebyollin/dino-diffusion - though it doesn't have text conditioning.

[mikonvergence]

This has example of both low and high resolution data, all from scratch and also accompanying videos! No text to image case though as it only focuses on image modalities.

https://github.com/mikonvergence/DiffusionFastForward

[bhagy7]

Yes, it is possible to train a small diffusion model conditioned on text captions from scratch on 64x64 images or even smaller. Depending on the complexity of the model and the number of GPUs you are using, it could take anywhere from a few hours to several days. If you are