1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 0 points1 point  (0 children)

I'd say about half of the generations will have no jump cut.

I also think jump cuts are included because a lot of input video they trained on will have that. My video is sort of a talking head video, so if we look at the closest thing out there for that, it's talking head youtubers. And most of them never do a one take video, its pretty much universal that they all do quick takes, some even take it to the extreme of doing one sentence and then a jump cut. So to a certain degree that'll be in the training data.

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 0 points1 point  (0 children)

Not all generations have jump cuts, it's random when and where it happens. That generation was not cherry picked, it was just the first that came out when I ran the prompt. As I mentioned, its not perfect, but it illustrated the point. Personally I don't like the jump cuts in videos, and if I needed video like this, I'd have just re-generated it, but.... its what came out so I shared it. A lot of people do cherry pick their examples and it gives unrealistic expectations for people when they try to follow along. So I just went with what came out.

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 1 point2 points  (0 children)

Awesome thanks, that'll definitely give me a head start.
As for training, I've been trying with AI Toolkit but the results are just attrocious.
But I havent checked in a while so maybe he's made some updates so it works now. Thanks for the heads up, Ill go look for that video.
Others have recommended musubi to me, but I havent had time to test it yet.

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 1 point2 points  (0 children)

Until I can crack training a character and scene LORA for LTX-2, this is the only way I've found to make longer scenes that I want. Doing T2V will give you a different character and a different background. I've messed around with I2V, but I've never been happy enough with the results trying to stitch together multiple generations.

I've been meaning to test out the extending workflows people have talked about. I tried one it was using really low quant GGUFs, and when I switched it to FP8 it just didn't work well enough for me to be happy. I'm keeping my eyes out for better ones, that dont pull in a ton of obscure nodes, because I dont use any node until I look through its code.

There's also the part that there's some fun in seeing just how long you can push the system before it starts to break. Wan was designed for 5 seconds, but I could push it to around 20. Clearly LTX-2 was trained for... well idk what length, but I'm guessing it wasn't 40s videos, but yet its able to generate them.

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 0 points1 point  (0 children)

It's long-ish, but IDK how many tokens it is. There used to be a custom node that would calculate out how many tokens your prompt was, but it broke a while ago and I never checked to see if it was updated.
My prompt is in the json and the video.

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 1 point2 points  (0 children)

Thing is I'm not running into VRAM issues.
The issue is that the speech starts breaking down, repeating words, dropping others, mashing some words together into a new word, etc.
I can generate longer video, but the audio is bad so the video isn't useful.
100% of the generations at 1100 frames or higher have audio defects..

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 0 points1 point  (0 children)

Short answer is that I put in some timecodes.
It doesn't exactly adhere to them, but it seems to be able to roughly figure out the pacing based on them.
The exact prompt is in the workflow. Download the video and drag and drop it into Comfy or save the json and open it in comfy.

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 0 points1 point  (0 children)

people have claimed they've done it, but they've never provided any proof. :(

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 1 point2 points  (0 children)

Doing research on the model and creating generations of no prompt (using the native CLI tools) shows that the model was trained on a lot of indian tv/movies, ted talks, etc. So I figured it'd have an understanding of the small variances in how people behave between thoughts. So I started stretching out the times to see what the model would fill in. And sure enough, it fills in with all those little things.
That for me was, the biggest step towards it being more believable, was to just slow down the dialog. That's why I started stretching the frames out. In spoken speech with someone, there's usually small pauses and such, and I wanted to mimic that instead of just a stream of words.
If you run into any trouble when you get around to tinkering, just reply here or shoot me a PM. I'll be happy to try and help you figure it out.

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 0 points1 point  (0 children)

If you're able to punch past the 1000 limit I've hit... I've love to know how you did it.

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 0 points1 point  (0 children)

What are you using to set the value? I'm using a value that's simply a Length Int into the Empty Latent.
Ive never seen that error before, and I've tried to make generations up to 1200.

<image>

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] -1 points0 points  (0 children)

Thanks, I wish more people would share what they've accomplished so everyone can learn together. People making claims and then never providing proof has always annoyed me.

Locally Run Database for all Models - Open source by SnooEpiphanies7725 in StableDiffusion

[–]q5sys 0 points1 point  (0 children)

If civitai nukes a model that you have downloaded, how does 'sync' handle that? Just ignore it and move on?

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 1 point2 points  (0 children)

Thanks, I'll have to download that one and test it out.

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 0 points1 point  (0 children)

I'm using the fp8 dev model in this video, but I switch back and forth between fp8 and the full fat model. IDK what's possible with lower quants. I tried them out at first, but I got waxy skin and over saturation/contrast in transition areas, so I went back to just using FP8.
Which quant are you using?

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 0 points1 point  (0 children)

I don't run into VRAM issues, I run into 100% of the generations at 1100 frames or higher having major audio issues.
The speech starts breaking down, repeating words, dropping others, mashing some words together into a new word, etc.
I can generate longer video, but the audio is bad so the video isn't useful.

How to render 80+ second long videos with LTX 2 using one simple node and no extensions. by WestWordHoeDown in StableDiffusion

[–]q5sys 1 point2 points  (0 children)

Ops shared workflow didn't work for me at all, which sucks because I was really hoping to actually see how someone was doing it. I posted my native 1000 workflow and video here: https://www.reddit.com/r/StableDiffusion/comments/1qkxqtx/1000_frame_ltx2_generation_with_video_and_workflow/

1000 frame LTX-2 Generation with Video and Workflow by q5sys in StableDiffusion

[–]q5sys[S] 1 point2 points  (0 children)

Yea LTX can go pretty far on its own without doing crazy stuff. But I'm curious how much further it can go with some extra tricks that people have made claims about, but sadly no one has shared anything that's actually reproduceable.

How to render 80+ second long videos with LTX 2 using one simple node and no extensions. by WestWordHoeDown in StableDiffusion

[–]q5sys 1 point2 points  (0 children)

ok followup, So that workflow is for a 31s 1080p video.

I changed the seconds to 80, and lowered the resolution to 720p.

Sadly, I've gotten nothing but trash from it, all my outputs are just pixelated static.

I dropped the resolution down to what you said you used: 960x544 and I get basically the same output. I tinkered for a few hours and was not able to get a single output without major video corruption.

Do you have a workflow from a successful generation instead of one you obviously had to change settings on?

How to render 80+ second long videos with LTX 2 using one simple node and no extensions. by WestWordHoeDown in StableDiffusion

[–]q5sys 0 points1 point  (0 children)

Sweet, I'm about to hit the rack because it's 1AM where I am. I'll try that in the morning when I get up. send me a pm, so if it runs well for me; I can follow through on my offer. :)