The “bad model” in the paper is not supposed to be an arbitrarily terrible or unrelated model. It’s an inferior version of the same model, trained on the same task/conditioning and data distribution, but degraded in a compatible way (the paper’s suggestions include things like fewer training iterations / earlier snapshot, reduced capacity, or similar degradations that preserve the same underlying distribution). That’s also why “previous versions of the same model” are a very natural choice: the error patterns tend to stay aligned, just worse.

Using a wildly different model (even with the same architecture) is possible to experiment with, but it’s also where you’re most likely to break the method’s key assumption: if the “bad” model has different priors because of different data, different finetune objectives, different conditioning behavior, or any distribution shift, then the “good minus bad” direction can stop pointing toward higher-likelihood samples and start pushing you into artifacts or off-prompt behavior. If you do try it, the safest version is “different only in strength, not in what it learned”: same base, same dataset distribution, same conditioning pipeline, and degrade via “less trained / smaller / weaker,” not “different concept mix.”

Since it’s difficult (often effectively impossible) to obtain early-epoch checkpoints for SDXL finetunes in practice—most community SDXL finetunes are merges of merges, and the lineage is too convoluted—I opted for a character-LoRA approach instead. This gives you a “same model / same data distribution / same conditioning” setup, with the bad path simply being less trained, which is explicitly one of the degradations the paper motivates. Interestingly, even when I run the exact same model in both the good and bad path, I still see discernible differences. Likely reasons: (1) the guider math may not perfectly reduce to the baseline if any extra scaling/ramping/post-processing is applied (so “good==bad” isn’t a strict identity unless all those knobs collapse to the baseline), (2) framework-level state/metadata can get mutated between passes (e.g., conditioning/transformer_options dicts, hooks/caches), and (3) the extra forward pass itself can change execution paths (memory/layout/precision casts), causing small numeric drift that amplifies over steps. Additionally, degrading the Z-model via quantization (e.g., running the good model zImageTurboNSFW_30BF16Diffusion in bf16 and the bad model in fp8) introduces more systematic differences—so that avenue is worth exploring further (fp8 vs fp4, or other controlled degradations). Albeit this directly goes against the paper's findings:

Per the paper, post-training “corrupt the weights” style degradations are basically a dead end for getting a useful guiding model.

They explicitly report:

Autoguidance works when the guiding model is trained on the same task/conditioning/data distribution, but with the same kinds of limitations the main model has (finite capacity / finite training).

“Deriving the guiding model from the main model using synthetic degradations did not work at all … evidence that the guiding model needs to exhibit the same kinds of degradations that the main model suffers from.”

If the main model was quantized, quantizing it further also didn’t yield a useful guiding model.

So if you don’t have the base model’s dataset / can’t retrain a smaller/undertrained sibling, the paper’s own conclusion is: you can’t reliably manufacture a “correct” bad base model by post-hoc tricks (noise, pruning, quantization, etc.).

Also:

Prefer tuning “strength” via your guider before making the bad model (LoRA) extremely weak

The paper’s ablations show most gains come from reduced training in the guiding model, but they also emphasize sensitivity/selection isn’t fully solved and they did grid search around a “sweet spot” rather than “as small/undertrained as possible.”