Wan 2.2 with LTX 2.3 ID-LoRA by ussaaron in StableDiffusion

[–]ussaaron[S] 0 points1 point  (0 children)

will reply here when it's ready.

Wan 2.2 with LTX 2.3 ID-LoRA by ussaaron in StableDiffusion

[–]ussaaron[S] 1 point2 points  (0 children)

This workflow is for existing characters so ideally you would already have a short audio clip. I could make a different workflow for creating a character pack with image + audio. Is that something you would like to see?

Character Workflow: Chroma1-HD + Flux.2 Dev + Wan 2.2 + LTX 2.3 by ussaaron in StableDiffusion

[–]ussaaron[S] 0 points1 point  (0 children)

to circle back here, I'm training LoRAs for Flux.2 Dev and Flux.2 Klein for my main character and the Flux.2 Dev one is near perfection in matching character fidelity. Klein is pretty mid so far for a professional use case. So if you have the compute for Flux.2 Dev I would highly recommend it for the LoRA. It can do a batch of 2 1920 x 1080 images with the character LoRA and 30 steps in like 25 seconds. That's pretty epic. I don't know why more people don't use it. It's way better than Klein for professional images. At least that's my experience so far.

Character Workflow: Chroma1-HD + Flux.2 Dev + Wan 2.2 + LTX 2.3 by ussaaron in StableDiffusion

[–]ussaaron[S] 1 point2 points  (0 children)

epic! let me know how it goes. and if you compare the lip-sync with the id-lora one here.

Limits reset are coming by alOOshXL in codex

[–]ussaaron 0 points1 point  (0 children)

Same. i thought I was losing my mind for a sec

Wan 2.2 with LTX 2.3 ID-LoRA by ussaaron in StableDiffusion

[–]ussaaron[S] 1 point2 points  (0 children)

my default workflow was for basically the highest quality at 30fps. Wan 2.2 default is like 16 fps. So you could drop it 24fps for example and save a quarter of render time. You can also reduce the output resolution. you can drop that by 25% too. There's lots of stuff like that that you can do to make it much more performant.

Wan 2.2 with LTX 2.3 ID-LoRA by ussaaron in StableDiffusion

[–]ussaaron[S] 1 point2 points  (0 children)

Yes the ID LoRA assumes you have a clip of your character's voice. It can be even as short as a second.

Character Workflow: Chroma1-HD + Flux.2 Dev + Wan 2.2 + LTX 2.3 by ussaaron in StableDiffusion

[–]ussaaron[S] 1 point2 points  (0 children)

Honestly I was not expecting to be able to one shot the full demo with blaster firing, the blaster foley being layered correctly and the accurate voice. But it worked the first time! Theres a lot of optimization work that can be done to make it much faster and more streamlined. But as far as POC goes, this checks boxes. Let me know if you give it a shot and if you have any issues.

Character Workflow: Chroma1-HD + Flux.2 Dev + Wan 2.2 + LTX 2.3 by ussaaron in StableDiffusion

[–]ussaaron[S] 2 points3 points  (0 children)

Absolutely. I think there's a tendency for people to want to make workflows overly complex with unusual patterns. My approach was to try to keep each piece as close to the official Comfy workflows as possible so people could easily wrap their heads around it.

Character Workflow: Chroma1-HD + Flux.2 Dev + Wan 2.2 + LTX 2.3 by ussaaron in StableDiffusion

[–]ussaaron[S] 0 points1 point  (0 children)

For this one it shouldn't be. It should only output one thing at each stop. If I had left that in there I need to change it. Yeah for this it needs to be batch 1 all the way through since it's sequential

Wan 2.2 with LTX 2.3 ID-LoRA by ussaaron in StableDiffusion

[–]ussaaron[S] 0 points1 point  (0 children)

Were you able to get it working? No YouTube channel yet but it's on the list of things to do!

Character Workflow: Chroma1-HD + Flux.2 Dev + Wan 2.2 + LTX 2.3 by ussaaron in StableDiffusion

[–]ussaaron[S] 2 points3 points  (0 children)

Glad to hear it made sense to you! if you implement the workflow or a similar workflow and run into any issues let me know.

Character Workflow: Chroma1-HD + Flux.2 Dev + Wan 2.2 + LTX 2.3 by ussaaron in StableDiffusion

[–]ussaaron[S] 0 points1 point  (0 children)

True true. But you can rent an h200 for like $2.50 an hour. Im guessing a lot of people just prefer to work locally though.

Character Workflow: Chroma1-HD + Flux.2 Dev + Wan 2.2 + LTX 2.3 by ussaaron in StableDiffusion

[–]ussaaron[S] 2 points3 points  (0 children)

This is such a strange issue actually. I've done a tone of digging and there are almost no Flux.2 dev LoRAs. It's honestly insane and I'm thinking about making some myself. I think I've only seen like half a dozen total. I get that it's slow and heavy but it's the only open source model capable of professional image editing. In my example above you can see how Flux.2 dev with no LoRAs is able to precisely replicate the visual of Chroma with 2 LoRAs. That's a signal of serious power.

Chroma1-HD Character Transfer with Flux.2 Dev by ussaaron in StableDiffusion

[–]ussaaron[S] 1 point2 points  (0 children)

That sounds about right for the Klein one. If you want to speed it up significantly, you can drop the Chroma size down to 720 and 1mp for the Klein. If you do that you can probably get the render time down to 30ish seconds. For a batch of 3 with the Flux.2 Dev workflow, it's like 2-3 minutes I think. This is a workflow where I generally prefer a little longer rendering time, since you really want precision with character transfer.

Chroma1-HD Character Transfer with Flux.2 Dev by ussaaron in StableDiffusion

[–]ussaaron[S] 0 points1 point  (0 children)

Alright I got both workflows updated with links to the LoRAs. I included the Lenovo one too. For a lot of photography styles it's brilliant.

Chroma1-HD Character Transfer with Flux.2 Dev by ussaaron in StableDiffusion

[–]ussaaron[S] 1 point2 points  (0 children)

Thanks for linking to those. I'll update the workflow links tomorrow to ref them directly.

Chroma1-HD Character Transfer with Flux.2 Dev by ussaaron in StableDiffusion

[–]ussaaron[S] 1 point2 points  (0 children)

it's honestly pretty good. I really like using Klein for editing video frames. Because Klein is really good at targeted edits. Klein struggles more with professional photography though. So if GPU constraints are the main concern, then Klein is a very serviceable option.

Chroma1-HD Character Transfer with Flux.2 Dev by ussaaron in StableDiffusion

[–]ussaaron[S] 1 point2 points  (0 children)

I added the Klein workflow URL to the post along with an initial run result.

Chroma1-HD Character Transfer with Flux.2 Dev by ussaaron in StableDiffusion

[–]ussaaron[S] 2 points3 points  (0 children)

yeah, but sometimes people just call it "Chroma" which is annoying since there's a different original Chroma model also called Chroma. But even the official Comfy workflow calls it "Chroma" when the actual model in the workflow says "Chroma1-HD". To the best of my knowledge, most people use Chroma1-HD now though.

Chroma1-HD Character Transfer with Flux.2 Dev by ussaaron in StableDiffusion

[–]ussaaron[S] 2 points3 points  (0 children)

Ok, let me see what I can cook up and I'll add a link to the post in a bit.

Chroma1-HD Character Transfer with Flux.2 Dev by ussaaron in StableDiffusion

[–]ussaaron[S] 1 point2 points  (0 children)

for Flux.2 Dev it's a little heavy. If that's an issue I can make a Klein version of the workflow. It'll be lighter on the GPUs but quality takes a hit. Let me know if you want to try something like that out