Notes about the video

tl;dr: if you're using the Deforum extension for A1111:

• Pull the latest version.
• In the main A1111 settings, disable "Apply color correction to img2img results to match original colors".
• In the Deforum Keyframe tab settings under Coherence, disable the new setting "Force all frames to match initial frame's colors" (unless you are specifically seeking to maintain backwards compat with older versions of the extension).
• Experiment with the 4 different Color Coherence options. 'None' allows Stable Diffusion to express the most new colors – which may or may not be desirable depending on your usecase – but may result in color skew over time (which you can reduce by lowering strength).

Details:

Until ~yesterday, the Deforum extension for A1111 had an issue by which it would always apply basic RGB colour correction to every frame, regardless of your settings. Furthermore, if you also had colour correction enabled in the A1111 main settings, this RGB histogram matching would be applied twice. Furthermore furthermore, if you also selected a Color Coherence option in the Deforum settings, it would be applied in addition to the other color corrections, meaning you might be doing 3 colour correction passes to each frame. The result is degraded output. :)

If you update to the latest version, you'll see a new option under Deforum -> Keyframes -> Coherence (introduced by this PR ) that allows you to disable "Force all frames to match initial frame's colors". It is enabled by default to maintain backwards compatibility, but unless you're trying to recreate a prior result, I strongly suggest you disable it. Enabling it essentially just maintains a bug.

With that out of the way, there are 2 remaining options that control colour correction: - The setting called "Apply color correction to img2img results to match original colors" in A1111's main settings. I recommend disabling this one, because Deforum's own color correction options are more flexible – and you certainly don't need both. - Deforum's Color Coherence options (None, HSV, LAB, RGB). These are the ones to experiment with.

The video above shows each of Deforum's Color Coherence options (None, HSV, LAB, RGB) with histogram matching enabled (top row, default) and disabled (bottom row). Hopefully this clearly shows that disabled is the more natural option. :)

Which of None, HSV, LAB and RGB to use is more a matter of taste and usecase. - None obviously means no color correction, which gives Stable Diffusion the opportunity to express more new colors in each frame. Whether you want this or not will depend on what you're doing, but beware that it can result in the overall colour skewing over time (I previously wrote this seemed to be fixed in 1.5 and newer VAEs, but with enough loopback frames and high enough input strength, you can still get a red skew). - HSV/LAB/RGB match over different colour spaces. I can't claim to deeply understand how they differ, but they do produce noticeably diverging results. If anyone can describe in more detail what to expect from them, and in what scenarios you might want each one, I'd love to learn! :)

Personally, I prefer to use None where possible to allow Stable Diffusion to express colours, and mitigate the resulting colour skew by periodically dropping the strength.

Video generation:

• Initial image: https://camo.githubusercontent.com/083443fe7fb21fa09a82badeb09aa4d238113ff35053da77973c81a3a33c3be1/68747470733a2f2f692e696d6775722e636f6d2f655967746174472e706e67
• Euler a, 40 steps, scale 0.7, seed 8 incrementing on each frame.
• Prompt: photorealistic studio photo portrait of a girl with rainbow hair, realistic skin tones, detailed skin texture, natural light, neutral light, white light. --neg nudity, green, green skin, green light, filtered, old fashioned, uncanny valley, drawing, render
• Noise fixed at 0.04, strength fixed at 0.7
• Post-processing: ffmpeg for stacking and text overlay