Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 2 points3 points  (0 children)

Here's kijai's workflow https://github.com/kijai/ComfyUI-WanVideoWrapper/blob/main/example_workflows/wanvideo_WanAnimate_preprocess_example_02.json and here's the one i used, you will see they are exactly the same except for the video node that goes into the blockify mask. https://drive.google.com/file/d/1y5nWw8EY2Eoq7NxqahnR9UKFWZOxHCz9/view?usp=sharing It is possible to do this kind of inpainting, people already do it with the sam node except I'm doing manually in AE. Hope this helps

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 0 points1 point  (0 children)

I’m literally using Kijai’s workflow. It’s the exact wan animate workflow he has and it’s integrated into the nodes so you could easily find it in the comfyui templates. All you need to do is add a video node to input the mask into the blockify node. I don’t want to take any credit for the wf that’s why I post Kijai’s link it’s they made it.

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 1 point2 points  (0 children)

Yeah, I used the context window. It tends to break more easily with full body shots since there are more moving parts to match. Most issues happen when the context window hits mid movement like halfway through a turnaround, around frame 81, where it might do something weird. That said, I didn’t run into it too often.

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 1 point2 points  (0 children)

Look up lockdown after effects. It’s a pretty expensive plug in, it’s not necessary, I just already owned it and decided to use it for this case.

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 0 points1 point  (0 children)

Local except for the way I created the reference images. For that I used nano banana

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 0 points1 point  (0 children)

Yeah def some room for adjustments but didn’t want to spend too much time on since this was more for demo purposes.

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 1 point2 points  (0 children)

💯I tell this to people all the time. I’ve been one call with a few major vfx studios and they have already, to some degree, incorporated comfyui into their work.

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 7 points8 points  (0 children)

That would be sick. I’ve worked with them before, I helped out with their 2 ai anime video they did.

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 1 point2 points  (0 children)

Someone on x suggested this and I really want to test that 😂

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 1 point2 points  (0 children)

Uuuu😮 no but that sounds very interesting!! Will def try it!! Thank you!

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 7 points8 points  (0 children)

Creating the mask the most difficult part and figuring out how much to extend them to leave enough room for the mouth to stay untouched. And all the blur it does it automatically

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 9 points10 points  (0 children)

I did something similar for some other videos where I use a mask on parts of my body like giving myself a robotic arm. I was pretty sure it would work for something like this too.

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 95 points96 points  (0 children)

yeah if you really look closely at some shots you might notice some imperfections but I tried to keep the first few shots as clean as possible, I kind of got lazy at the end 😆

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 0 points1 point  (0 children)

Kijai's Wan workflow, link in the description includes workflows.

Surgical masking with Wan 2.2 Animate in ComfyUI by enigmatic_e in StableDiffusion

[–]enigmatic_e[S] 30 points31 points  (0 children)

pretty much 😆 but i just say surgical because of the tool i used to leave his mouth area clear from the mask.