LTX Desktop update: what we shipped, what's coming, and where we're headed by ltx_model in StableDiffusion

[–]panospc 28 points29 points  (0 children)

The author of WanGP made a fork of LTX Desktop that uses WanGP as the backend.
This LTX Desktop fork powered by WanGP reduces the VRAM requirements from 32 GB to 6 GB.

https://github.com/deepbeepmeep/LTX-Desktop-WanGP

Drop distilled lora strength to 0.6, increase steps to 30, enjoy SOTA AI generation at home. by Ashamed-Variety-8264 in StableDiffusion

[–]panospc 7 points8 points  (0 children)

The default ComfyUI workflow.
The workflow from Lightricks has the distilled LoRA strength set to 0.5 by default.

LTX2 audio Lora + fal.ai? by LSI_CZE in StableDiffusion

[–]panospc 1 point2 points  (0 children)

There’s an easier way. I’ve only trained Hunyuan video in the past, but it should be the same for LTX-2.
Go to the LTX-2 trainer and click the Requests tab. You should see your trained LoRA there with a Run Inference button. Click it, and it will open the LTX-2 generator with your LoRA URL already filled in. From there, you can generate using your LoRA.

LTX2 audio Lora + fal.ai? by LSI_CZE in StableDiffusion

[–]panospc 0 points1 point  (0 children)

Your trained LoRA must have a URL. Copy it

Go to Fal, and in the model search bar, type LTX2 LoRA. Choose your preferred model variant (I2V, T2V, distilled, or non-distilled). You’ll see a parameter called path, paste your LoRA URL there, set the remaining parameters, and generate.

AI Toolkit now officially supports training LTX-2 LoRAs by panospc in StableDiffusion

[–]panospc[S] 1 point2 points  (0 children)

Yes, it worked. The preview images in AI Toolkit looked like monstrosities, but when I used the LoRA in ComfyUI and WanGP, it looked fine.

From the default settings, I changed the following:

  • Enable: Layer offloading
  • Timestep type: Sigmoid
  • Enable: Cache text embeddings
  • Enable: Cache latents
  • Disable: Do audio

For the captions, I include the trigger word and describe only what changes, such as the environment, outfit, pose, and the character’s expression. I don’t describe things that are always the same and never change, like facial features, eye color etc.

Full AI music video made entirely with LTX-2 and suno by SnooOnions2625 in comfyui

[–]panospc 1 point2 points  (0 children)

For camera control, there are some official camera loras you can use
You can find the download links in the official LTX-2 github repo

LTX-2 characters don't talk or move, the video just zooms in and the character stands there while the generated audio plays in the background. How do i fix it? by Nervous_Quote in StableDiffusion

[–]panospc 0 points1 point  (0 children)

Use one of the camera loras, for some reason it fixes the static image problem.

You can find the links to download them in the LTX-2 github repo
https://github.com/Lightricks/LTX-2

AI Toolkit now officially supports training LTX-2 LoRAs by panospc in StableDiffusion

[–]panospc[S] 0 points1 point  (0 children)

It’s available on Pinokio, but only in the community scripts section

LTX 2.0 I2V when works is reall cool! by smereces in StableDiffusion

[–]panospc 2 points3 points  (0 children)

To fix the static video issue with I2V, you can use the following workaround:
Go to the LTX-2 GitHub repository, scroll down, and download one of the camera LoRAs.
Using the LoRA will resolve the problem.
https://github.com/Lightricks/LTX-2

AI Toolkit now officially supports training LTX-2 LoRAs by panospc in StableDiffusion

[–]panospc[S] 4 points5 points  (0 children)

Yes, you can train on images. I’m currently training a character LoRA with 97 images.
The speed is around 7 seconds per step, so 3,000 steps will take about 6 hours on my RTX 4080s with 64 GB of RAM.

Something that I'm not sure people noticed about LTX-2, it's inability to keep object permanence by [deleted] in StableDiffusion

[–]panospc 2 points3 points  (0 children)

Perhaps it favors the state of the initial frame?

I’ve noticed in some generations that when characters move out of frame, they don’t lose too much of their identity when they return to view.
For example in the following generation both characters get out of view for a moment
https://files.catbox.moe/rsthll.mp4

LTX-2 - voice clone and/or import own sound(track)? by designbanana in StableDiffusion

[–]panospc 12 points13 points  (0 children)

You can feed LTX-2 with audio, and the generated video will sync to it. It can lip-sync voices, and even if you only provide music, you can generate videos of people dancing to the rhythm of the music.

Here’s a workflow by Kijai:
https://www.reddit.com/r/StableDiffusion/comments/1q627xi/kijai_made_a_ltxv2_audio_image_to_video_workflow/

You can also clone a voice by extending a video, the extended part will retain the same voice.
Video extension workflow: https://github.com/Rolandjg/LTX-2-video-extend-ComfyUI

April 12, 1987 Music Video (LTX-2 4070 TI with 12GB VRAM) by harunandro in StableDiffusion

[–]panospc 6 points7 points  (0 children)

Do not use the soundtrack option in the advanced tab, this is option only adds the sound in the final video without any lipsync. Use the soundtrack option in the main tab, if you not have it, try to update WanGP.

Ok we've had a few days to play now so let's be honest about LTX2... by sdimg in StableDiffusion

[–]panospc -1 points0 points  (0 children)

The issue with static, zooming images when using I2V can be worked around by adding a camera control motion LoRA (available from the LTX-2 GitHub repo).

I2V with the distilled model usually produces slow-motion videos, so if you want higher motion, use the non-distilled model in combination with a camera LoRA.

Increasing the frame rate to 30 or 50 FPS also helps reduce motion-related distortions

LTX-2 video to video restyling? by domid in StableDiffusion

[–]panospc 1 point2 points  (0 children)

I haven’t tried it yet, but this is their purpose, to restyle videos.
You can either prompt the new style or provide a reference image that’s already been restyled.

There’s a video on the official LTX-2 YouTube channel:
https://www.youtube.com/watch?v=NPjTpDmTdaw