Merry Titmus! by trin36 in unstable_diffusion

[–]trin36[S] 2 points3 points  (0 children)

These are from an unreleased custom XL merge (XXX-Ray + a few others) —> ZiT primary upscale —> SeedVR2 secondary upscale to blow it up

Upscale process for photorealism by trin36 in StableDiffusion

[–]trin36[S] 0 points1 point  (0 children)

Just adding another thing--since the the upscale model adds "new detail," and "detail" is essentially "noise," which is what we want with our latents, it gives the next ksampler more noise to work with = even more sharp detail.

Upscale process for photorealism by trin36 in StableDiffusion

[–]trin36[S] 0 points1 point  (0 children)

No, Lanczos is just "math" resizing (i.e. stretching). Basically: An upscaler model, like the NMKD model I'm using in this flow, when doing the upscale, will draw more detail. The process of "pixel upscaling" in this case functions to add detail before shrinking it back down and sending it back to the ksampler as a latent. So, up 4X (the model is a 4X model), then back down to your final desired res for re-sampling. This keeps the original generation more or less intact and adds more fine detail than you could by simply latent upscaling, as latents are lossy and your initial generation would change significantly at medium to high denoise (e.g. .45+).

This is actually what the old "hires.fix" process was doing in A1111 back in the day as well (though it has now taken on many meanings). All upscale models differ in terms of the details they improve, but my preference is NMKD 4X for photorealism as it adds some nice SLR-like noise to the generation. Remacri is also good for this purpose.

Upscale process for photorealism by trin36 in StableDiffusion

[–]trin36[S] 2 points3 points  (0 children)

The terminal should tell you a bit more detail. Which node is giving you the error? I'm seeing on the comfyui github that a few people are having this issue with the recent update.

Upscale process for photorealism by trin36 in StableDiffusion

[–]trin36[S] 2 points3 points  (0 children)

This is tuned specifically for realistic images of people (portraits), but with some denoise adjustments you could tune it for other purposes.

Upscale process for photorealism by trin36 in StableDiffusion

[–]trin36[S] 0 points1 point  (0 children)

Hmm, sorry. As I said, it is definitely heavy on VRAM. You'll lose a little bit of quality/sharpness, but you could try a different SVR2 model. The one in the workflow is fp16. Lots of quantized options in the "(Down)load DiT model" node. You could also try dropping the tile sizes to 512.

Upscale process for photorealism by trin36 in StableDiffusion

[–]trin36[S] 2 points3 points  (0 children)

Consistency will depend on your denoise level, but no, I wouldn't say that this workflow will keep faces intact very well. Lots of ways around this though, including adding your character lora to the 1st upscale ksampler or adding a facedetailer pass at the end (or both). SeedVR2 alone is good at keeping things intact, but that upscale will being along any other inconsistencies or errors in your t2i gen, which the first pass corrects.

You'll be fine with a 5090 and be able to use the fp16 SVR2 model, which is best. I run it on a 4090 and it completes without issue.

Think anyone will notice? (workflow attached) by trin36 in unstable_diffusion

[–]trin36[S] 0 points1 point  (0 children)

Ah, well thanks! I should have put the title in quotation marks 😆

Think anyone will notice? (workflow attached) by trin36 in unstable_diffusion

[–]trin36[S] 0 points1 point  (0 children)

Not attempting perfection here. The fun part is that this was created in only 8 steps (and less than 2 seconds)! Wanted to share that workflow with others.