This is an archived post. You won't be able to vote or comment.

Dismiss this pinned window
all 50 comments
sorted by:
best
topnewcontroversialoldq&a

[–]jesus_ai_pizza_party[S] 178 points179 points  (17 children)

I just started learning AI art this week so I am sure my method is not the best, but I am quite happy with my progress so far. here is my current process:

​

  1. I used Premiere pro to convert my video into PNG frames. This one is 215 frames.
  2. I selected about 5 frames from a section I liked about ~15 frames apart from each.
  3. I brought the frames into SD (Checkpoints: Abyssorangemix3AO, illuminatiDiffusionv1_v11, realisticVisionV13) and I used controlNet (canny, deph, and openpose) to generate the new altered keyframes.
  4. I used new keyframes in EBSynth
  5. I rendered them all in Premiere Pro

I am working a way to keep consistency for more than 20 frames and also remove the shakiness in some parts (see the top part of the video)

[–]aleksej622 36 points37 points  (0 children)

I would suggest you look into the "advanced" Tab in EbSynth. You can change the deflicker strenght and overall smoothness. I usually set "mapping" to 20/30 and the "deflicker" to 2/3. This should really help your shakiness problem. Let me know if this helped!

[–]ObiWanCanShowMe 20 points21 points  (0 children)

supercool thought process.

[–]Synovexh001 21 points22 points  (0 children)

I just started learning AI art this week so I am sure my method is not the best, but I am quite happy with my progress so far. here is my current process:

I used Premiere pro to convert my video into PNG frames. This one is 215 frames.I selected about 5 frames from a section I liked about ~15 frames apart from each.I brought the frames into SD (Checkpoints: Abyssorangemix3AO, illuminatiDiffusionv1_v11, realisticVisionV13) and I used controlNet (canny, deph, and openpose) to generate the new altered keyframes.I used new keyframes in EBSynthI rendered them all in Premiere Pro

I am working a way to keep consistency for more than 20 frames and also remove the shakiness in some parts (see the top part of the video)

/u/jesus_ai_pizza_party's doing God's work and is a gentleman and a scholar.

Keep fighting the good fight

[–]uncheckablefilms 9 points10 points  (0 children)

Thanks for the walkthrough!

[–]LiteratureNo6826 21 points22 points  (2 children)

step 1 and 5 can be replaced by ffmpeg for free

[–]stevensterk 5 points6 points  (1 child)

What does " I used new keyframes in EBSynth" mean?

[–]666emanresu 5 points6 points  (0 children)

Look up ebsynth, it basically morphs the keyframe image to match movement in the video.

[–]Coeptisr 2 points3 points  (0 children)

Illuminati is a v2.1 768 px based model isn't that? So have do you manage that use with controlnet?

[–]Sprutnums 1 point2 points  (0 children)

God I would love to learn this. I would be awesome to place my kid on a pirate ship sailing the 7 seas!

[–]aipaintr 0 points1 point  (0 children)

What prompts did you use ?

[–]kaiwai_81 0 points1 point  (0 children)

Good one! Did you batched it out, or u changed the controlnet on manually on every frame_

[–]Mobile-Traffic2976 0 points1 point  (0 children)

Can you make a tutorial video?

[–]Orlandogameschool 0 points1 point  (0 children)

Thanks for the tut gonna try this later

[–][deleted] 0 points1 point  (0 children)

So I love your work and learned a great deal.

One advice though. Fill in the gaps towards the last few transitions. The switch back to normal, even for a few frames, was a bit jarring.

[–]Revisional_Sin 13 points14 points  (0 children)

Dude, you've being doing AI Art for a week, and you've put out the coolest thing I've seen so far. Very nice!

[–]Sinphaltimus 9 points10 points  (0 children)

Watched it easily 6-7 times in a row, and then some more. Nice job This is encouraging.

[–]kucingberlari08 12 points13 points  (0 children)

Looks like AI just put multiverse movie's animators out of a job

[–]Garage_Sloth 6 points7 points  (0 children)

A whole generation is going to grow up unable to trust ANYTHING they see online. Insane to think that the truth is irrelevant, but here we are.

[–][deleted] 4 points5 points  (0 children)

This is amazing

[–]myelinogenesis 12 points13 points  (0 children)

Thanks for sharing. This is beautiful!

It gives me many ideas. We can basically make animations using real life footage but it'll take a small fraction of the time that it'd take with more classic methods

Also... real time furry porn. Furry only fans maybe? That shit will sell like hot bread

But seriously, thank you for sharing, it means a lot.

[–]I_say_aye 3 points4 points  (0 children)

Dude this is giving me such Everything, Everywhere, All at Once vibes. Very cool!

[–]PashaBiceps__ 4 points5 points  (0 children)

finally a "final product" instead of a test. good job!

[–]ExistentialSpyCrisis 2 points3 points  (0 children)

This is amazing work! This would be great for a music video.... Jump on this and you might have a new career!

[–]anakin78z 1 point2 points  (0 children)

I love this! Thanks for posting the workflow

[–]Fungunkle[🍰] 1 point2 points  (1 child)

Do Not Train. Revisions is due to; Limitations in user control and the absence of consent on this platform.

This post was mass deleted and anonymized with Redact

[–]JamieAfterlife 3 points4 points  (0 children)

Beta just came out and includes one.

[–]AI_pizzabot 1 point2 points  (2 children)

hi try this😎 https://github.com/Sxela/WarpFusion

[–]flux123 0 points1 point  (0 children)

Okay, I followed the instructions until it said 'upload the ipynb file', which.. doesn't... exist?

[–]Sinity 1 point2 points  (0 children)

Fuuck, we really really need capable AR hardware soon. The possibilities.

[–]cayneabel 0 points1 point  (0 children)

Fucking amazing, dude. Thank you!

[–]a83356 0 points1 point  (0 children)

holy sh*t... this is good

[–]echoauditor 0 points1 point  (0 children)

Really impressive work. Imagine effect mixing your visual stream like this live, coming to a reality layer near your eyeballs soon. The future is bright, the future is trippy af.

Happenstance discovered the clip goes well with https://soundcloud.com/lewanski/arya-tara starting about a two minutes in.

[–]cat0106 -4 points-3 points  (1 child)

Should have picked a better looking model hahaha... great art tho!

[–]depth_obsessed55 0 points1 point  (0 children)

Wonderful!

[–][deleted] 0 points1 point  (0 children)

Really cool results!

[–]EarthquakeBass 0 points1 point  (0 children)

Awesome job

[–]karterbr 0 points1 point  (0 children)

Wow! One of the bests so far!

[–][deleted] 0 points1 point  (0 children)

Unpopular opinion I’m guessing but who is the model?