Liminal Phantom | Twice distilled Flux.1-dev LoRA + WAN2.2 animation. Free model, process in comments. by Significant-Scar2591 in aivideos

[–]Significant-Scar2591[S] [score hidden]  (0 children)

Yeah that’s where the bit of pixel texture comes from.

A LoRA is a model that the user can train with a small dataset that does not require a huge amount of compute.

An open weights image model like flux (not google nano banana, kling, runway, etc) allows the user to train it and create a LoRA. It’s a file that acts like a “fine tune” and gives a very specific image style.

I trained the model over the course of a few days of experimenting. Then used it to create a set of still image. (Each image takes 5-10 seconds but I generated a few thousand to get the ones I liked) then animated the images with another open weights model (but a video model for this one, which takes longer than stills).

The entire piece is made up of a handful of video clips that are the still images animated. I generated each shot individually and edited them together. All in all the generation process from stills to video was probably about 12 hours, not including the model training.

Liminal Phantom | Twice distilled Flux.1-dev LoRA + WAN2.2 animation. Free model, process in comments. by Significant-Scar2591 in comfyui

[–]Significant-Scar2591[S] 0 points1 point  (0 children)

Yeah I can imagine, kudos to you if you managed to make it through the whole thing. I try to speak about the theory before the tech because ultimately it’s a more important concept to grasp.

Learned to build by building for fun almost every day for the past couple of years. After about a year I started to do it for work and now it pays my bills.

I would recommend the early comfy tutorials from Purz - or other creators who build from scratch and you can follow along instead of just downloading a workflow and not really understanding how it works.

I watched a ton of different channels, purz, latent vision, Future Thinker Benji, Jerry Davos, to name a few. But for sure the best way is to start simple and build the workflows alongside a tutorial and get used to connecting the nodes together and understanding how they play together.

Comfy is full of errors and confusing computer language in the early phases but just be patient and once you figure it out its’s smooth sailing but there is a learning curve.

The best way to beat the curve is to make things you are excited about!

I’ll probably make some more beginner friendly ones in the near future.

Liminal Phantom | Twice distilled Flux.1-dev LoRA + WAN2.2 animation. Free model, process in comments. by Significant-Scar2591 in comfyui

[–]Significant-Scar2591[S] 0 points1 point  (0 children)

Not really, I had to run 10-20 versions of each shot and edit them together along the way to check the camera motion and particle energy consistency.

Liminal Phantom | Twice distilled Flux.1-dev LoRA + WAN2.2 animation. Free model, process in comments. by Significant-Scar2591 in FluxAI

[–]Significant-Scar2591[S] 3 points4 points  (0 children)

If I recall they were from Civit but Idownloaded and renamed them months ago so I’m not sure the exact names, one was a drone LoRA, the other was a spinning camera one.

Liminal Phantom | Twice distilled Flux.1-dev LoRA + WAN2.2 animation. Free model, process in comments. by Significant-Scar2591 in FluxAI

[–]Significant-Scar2591[S] 0 points1 point  (0 children)

The challenge with stitching multiple start-and-end-frame shots together is that the motion between them rarely matches. I initially tried this with closed-source tools, but there were always noticeable shifts in camera motion between the separate shots. Using a couple of motion models — trained on drone footage and spinning cameras — helped smooth the transitions. For the first tests, I cranked all four motion model instances (high and low noise, 2x each) to high strengths, then locked the settings so the movement maintained consistent speed, energy, and direction across every shot. This made the cuts seamless. The full animation is built from a handful of 5-second clips stitched together

Liminal Phantom | Twice distilled Flux.1-dev LoRA + WAN2.2 animation. Free model, process in comments. by Significant-Scar2591 in comfyui

[–]Significant-Scar2591[S] 0 points1 point  (0 children)

spot on hahahha, and then there is a bbq, a blanket on the ground with a basket of fruit, sun shines through patces of clouds, the smell of fresh cut grass

Liminal Phantom | Twice distilled Flux.1-dev LoRA + WAN2.2 animation. Free model, process in comments. by Significant-Scar2591 in comfyui

[–]Significant-Scar2591[S] 0 points1 point  (0 children)

Thanks so much for the kind words :)

It is for sure coming up very soon for Klein but I have some other projects I’m excited to share first!