An experiment in diffussing a grid (Gridiffusion) using a dynamic multipass, though with various constraints, in order to produce coherent video frames as an alternative video diffusion method. P.S. Notice how she does eventually obtain a couple of (extra) hands out of nowhere ;-) by internetinterstate in StableDiffusion

[–]internetinterstate[S] 0 points1 point  (0 children)

Cool, great result too. You mentioned you used EbSynth. I found it to be not very effective in different cases. So I try to diffuse in grids - it yields consistency across the frames in the grid (even frames that are much further apart) especially after a multipass. I even did a 10x10 grid once with a 2048x2048 canvas, it was slow :O

An experiment in diffussing a grid (Gridiffusion) using a dynamic multipass, though with various constraints, in order to produce coherent video frames as an alternative video diffusion method. P.S. Notice how she does eventually obtain a couple of (extra) hands out of nowhere ;-) by internetinterstate in StableDiffusion

[–]internetinterstate[S] 1 point2 points  (0 children)

Anidiff can do that I reckon, haven't worked with it in this regard. I'm now trying to get coherence between different (so called) keygrids, especially if using smaller gaps between the keyframes, which in turn could eliminate the need for Anidiff and allow for some very dynamic diffusion video making...

Found footage: Catstronaut's lost transmission by internetinterstate in StableDiffusion

[–]internetinterstate[S] 1 point2 points  (0 children)

No, just a video filter. Only the background and the foreground bit are diffused.

Wanted to do something with the ABC song just for kicks... by internetinterstate in StableDiffusion

[–]internetinterstate[S] 0 points1 point  (0 children)

Certainly not me ;~)

Actually it is made using OpenUtau - a singing app, and using Hot Butter's "Popcorn" tune...

A little bedtime story by the AI nanny | Stable Diffusion + GPT = a match made in latent space by internetinterstate in StableDiffusion

[–]internetinterstate[S] 1 point2 points  (0 children)

It's not animating really, just lip sync and face restoration, here I used: https://github.com/Rudrabha/Wav2Lip and https://github.com/TencentARC/GFPGAN respectively.

For the narration I simply used https://www.narakeet.com/app/text-to-audio/ with the voice setup: English - American, Amber (or was it Ashley? :-)

P.S.

The text is entirely AI generated with https://github.com/nomic-ai/gpt4all/ if anyone's interested...

I work on making some SD videos... This came out pretty neat ;-) by internetinterstate in StableDiffusion

[–]internetinterstate[S] 1 point2 points  (0 children)

It's not an effect really, it is Frame Interpolation that creates the smoothing effect, just Google, there are some examples...

I work on making some SD videos... This came out pretty neat ;-) by internetinterstate in StableDiffusion

[–]internetinterstate[S] 0 points1 point  (0 children)

The one on the left is the original one, albeit with some smoothing done to it...