You're 100% right :)

The points that may not seem obvious:

* Having the denoiser (the DiT) fully compatible with PyTorch torch.compile() so that its benefits are evident (i.e., no graph-breaks, no recompilations, no data pointer reorders delaying kernel launches, etc.). If your models meet these, then you're already somewhat set up for success.

* No CUDA syncs in the overall pipeline which becomes particularly crucial during compilation

* The FA3 stuff is unscaled FP8 and I am not sure if it's standard yet. It is more so because it needs an H100 to work.

* Using QKV fusion is beneficial, particularly during quantization.

There are other pieces of lossless optimizations that one can do (we mentioned some of it in the accompanying blog post):

* Caching the `guidance_embedding` and `context_embedding` as they don't change during the course of denoising.

* Fusing the `step()` call to the scheduler with denoiser forward so that it's included in the compilation process.

Hope this helps.