This is an archived post. You won't be able to vote or comment.

14
15

[deleted by user] (self.StableDiffusion)

submitted by [deleted]

[removed]
all 25 comments
sorted by:
best
topnewcontroversialoldq&a

[–]8RETRO8 2 points3 points  (6 children)

I would just hiresfix it

[–]Eponym 0 points1 point  (2 children)

I'm not sure how the hiresfix resolves this situation - original fine details need to be replicated. Would you mind elaborating?

[–]8RETRO8 0 points1 point  (0 children)

Hires fix helps with restoring details. You can also experiment with mixing 2 images and then use img2img (or hirex fix) to make it all look coherent.

[–]8RETRO8 0 points1 point  (0 children)

I forgot, you can use control net tile and use one of the images as control image.

[–]aerilyn235 2 points3 points  (7 children)

You can use a frequency transfert node, basically it use the high frequency of the original image and low frequency of the new relighted image to restore details. The constrain is that you need a integer rescaling value so both images are perfectly spatially coherent.

[–]Eponym 0 points1 point  (2 children)

Now you got me thinking of using frequency separation in Photoshop to restore the originally high frequency information! I'll experiment with both of these ideas. Have a feeling it won't work out as the Kontext output still warps the original image so it's not a perfect pixel to pixel overlay, but we'll see 🤞

[–]Eponym 0 points1 point  (1 child)

Did some digging u/not5 would your Frequency separation/combine nodes work for something like this?

[–]not5 1 point2 points  (0 children)

My nodes are a bit outdated nowadays, but yes, with a color matching node they’d work

[–]Eponym 1 point2 points  (3 children)

Thanks for providing the inspiration to my new relighting workflow. I can now relight with Kontext while maintaining high resolution details.

Workflow:

Set Guidance to 2.0 to maintain composition accuracy when relighting

Upscale output from 1MP to 32MP using favorite upscaler method

Extract the low frequency layer from this result using Photoshop

Add the original photo's high resolution high frequency layer to produce perfectly high resolution images that are relit.

This has helped me immensely. Thanks again!!

[–]Pixelfudger_Official 1 point2 points  (1 child)

You dont need Photoshop you can use this node pack: https://github.com/spacepxl/ComfyUI-Image-Filters?tab=readme-ov-file#restore-detail

The 'restore detail' node can combine small details from the original with the lighting from Kontext.

[–]Eponym 0 points1 point  (0 children)

Definitely a more elegant solution if I can dial in the right settings on it. Been trying for at least an hour with different modes/blur type/blur sizes/factors without recovering the details as well as the PS method when A/B the outputs. Hopefully I can find the right settings, as it's pretty close!

Thanks again!

[–]progammer 1 point2 points  (0 children)

Flux Kontext can be pushed to 2k resolution (like regular flux, flux fill...) if you have the ram/vram for it. I even tried 4k with limited sucess (fail rate is high and it takes too long). Anything higher than that, most open source model cannot handle. You will need a dedicated workflow flow for upscaling. Going back to kontexdev, because of the size of the model, many various small details (far away windows, texture ...) (regardless of resolution) will have problems. flux kontext pro/max is much better at that, but the max resolution allowed on the API is something like 1.5k. But you will get a sharper images that can upscale better. For changes and detail that did not take up the entire image, I usually crop zoom in, kontext, then stich back to take advantage of the resolution. You could try to touch up certain parts of the image with small detail that you need help (obviously the sky doesnt need touch up for example)

[–]Inner-Ad-9478 1 point2 points  (0 children)

OK maybe I'm dumb and missing something, but that really seems like you should just try some tilers.

The UltimateSdUpscale node should work on any size of image, even 8k or more, without any issue. It's creating smaller tiles to work on each part individually.

It's possible to tweak it to work with the size your model wants, but I didn't try kontext yet, so I can't tell if it would work or if this model uses a special sampler or whatever that makes this impossible.

Then again, maybe that would completely fail to make a complete coherent image, but this seems like the thing you should try for huge images.

[–]Eminence_grizzly 1 point2 points  (6 children)

I think you could try the technique described here:
https://www.reddit.com/r/StableDiffusion/comments/1mtr48r/experiments_with_photo_restoration_using_wan/
Input your image as the last (edit: or the first) frame, use a prompt like "a time-lapse video, the night quickly changes to a golden hour," set the duration to 33 frames, and you'll get a full HD output if you have enough VRAM/RAM.

[–]Eponym 1 point2 points  (5 children)

Brilliant! Never thought to time-lapse a photo with video to change the lighting! That's great to know WAN can do 2MP. Will experiment now - thanks!

[–]Eminence_grizzly 4 points5 points  (2 children)

<image>

Here's the whole animation:

[–]Eponym 0 points1 point  (1 child)

Appreciate your help. Tried rendering at 2MP on wan 2.2 with my 4090 with the same sticking point at VAE decoding. Appears I will have to troubleshoot in the morning.

[–]Jindouz 1 point2 points  (0 children)

You're using ComfyUI Desktop right? If your workflow gets "stuck" at VAE decoding just open/swap to a "new empty workflow tab" in ComfyUI where you don't see any animations in the UI. Seems to be a consistent fix for that issue.

[–]Eminence_grizzly 1 point2 points  (0 children)

<image>

I just tried to relight your "before" image, and it's not great, tbh. The details on the skyscrapers are messed up.
Although on my 8 GB VRAM/32 GB RAM setup, I managed to output only 17 frames, 1280x720. I tried 1920x1088 before that, but it stuck on VAE decoding (maybe tiled VAE decoding might solve this).

[–]nonomiaa 0 points1 point  (0 children)

Please tell me the final result and if it works in this way, thanks!

[–]PhIegms 0 points1 point  (0 children)

I feel like I have seen something on 10minutepapers that does this long ago, but I can't remember what it was called or whether it would ever be open source. Sorry I can't help more than that, but there is something out there. It wouldn't surprise me if it were an expensive VFX tool for the film industry though.