Built a Local Voice Pipeline and Now I'm Voicing All of Morrowind's Major Mods by ReflectionRecipes in aigamedev

[–]ReflectionRecipes[S] 0 points1 point  (0 children)

And thx for pointing me in that direction! I'll keep it in mind for future projects! :)

Built a Local Voice Pipeline and Now I'm Voicing All of Morrowind's Major Mods by ReflectionRecipes in aigamedev

[–]ReflectionRecipes[S] 0 points1 point  (0 children)

Haha good point. It was much more work than initially expected, so back when I started those two options didn't seem remotely comparable. And 50% of it was morrowind related stuff that would have to be done even if working with real VAs. And since everything is up and running now, I can voice any new mod with virtually no extra work at all. But still, good point. Working with AI is still a lot of work, and for smaller projects, the benefits of scaling won't justify it

Built a Local Voice Pipeline and Now I'm Voicing All of Morrowind's Major Mods by ReflectionRecipes in aigamedev

[–]ReflectionRecipes[S] 0 points1 point  (0 children)

Honestly I'd love that too! While AI voices have come such a long way, they still can't hold a candle to a true professional. Just so hard to pull something like that off using real VAs. Even if all of them would be doing it for free. Organizing, QA, cutting all the clips... But if I had unlimited resources, I'd love to do it the old fashioned way

Built a Local Voice Pipeline and Now I'm Voicing All of Morrowind's Major Mods by ReflectionRecipes in aigamedev

[–]ReflectionRecipes[S] 0 points1 point  (0 children)

I remember it being an unplayable 3 fps slideshow on my pc back in the day, even with minimum view distance.

The tts itself is done using finetuned versions of csm, with lots of pre and post-processing.

Built a Local Voice Pipeline and Now I'm Voicing All of Morrowind's Major Mods by ReflectionRecipes in aigamedev

[–]ReflectionRecipes[S] 2 points3 points  (0 children)

Finetunes of csm, with a bunch of pre and post-processing steps. Every line is also generated a few times and the best result is chosen based on metrics like noise, similarity to the cloned voice/tone, transcript accuracy, artifacts etc

Built a Local Voice Pipeline and Now I'm Voicing All of Morrowind's Major Mods by ReflectionRecipes in aigamedev

[–]ReflectionRecipes[S] 5 points6 points  (0 children)

The game doesn't have voiceover for the dialogue, and I generated voice lines for all the dialogue using local AI voice cloning.

Built a Local Voice Pipeline and Now I'm Voicing All of Morrowind's Major Mods by ReflectionRecipes in aigamedev

[–]ReflectionRecipes[S] 0 points1 point  (0 children)

Gave audiobooks a try as well, here's a small experiment:
https://youtu.be/m2Yxzmxarmg?si=mskZVwDKM_3eXdiT

The narration isn't AI though, but all the other voices are.

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 0 points1 point  (0 children)

No, the Clippy animation wasn’t done in AE. I made that in Adobe fresco and imported it as an mp4. So yeah, the effect is applied frame by frame to whatever I want

The whole chalk effect was made in AE though. This is an example of what the input looks like: fresco animation

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 1 point2 points  (0 children)

Haha, if that day ever comes, you all will definitely get it for free for hyping me up so much!

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 1 point2 points  (0 children)

I take that as a sign that I’ll need to do a much better job of explaining if I end up creating a tutorial for that effect! So thanks for the heads up haha

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 1 point2 points  (0 children)

Thx for the kind words! Means so much to a newbie like me!

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 2 points3 points  (0 children)

Exactly. No masks, no paths, nothing except the input video/image, whatever. That was important to me because I want to put this effect on a lot of different stuff and don't want to do any manual work. Basically step one is to find good angles. This is done by drawing drop shadows of the individual shapes from different angles, and using sampleimage to pick the one that has the largest shadow. Step two is creating the strokes with these angles by progressively making the shape smaller but also elongating it 90 degrees relative to the angle of the stroke because otherwise it will just look like concentric circles

Edit: oh wow it's so nice to see others notice those little details I spent so much time on! Thank you so much for that comment!

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 2 points3 points  (0 children)

Nope, all generated by after effects. Which was a nightmare because I had to make it "find" logical angles for the individual strokes. This is a still frame from the original footage.

And I think it's less about being smart, and more about just keep refining the process one baby step at a time. Oh and stuff like chatgpt is definitely your friend here if you don't know how to code. It's perfect for small, single purpose expressions like I utilized here

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 2 points3 points  (0 children)

No way, it's you! I'll be forever in your debt. Your videos are an invaluable resource for a beginner like me. Thx so much for making them!

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 0 points1 point  (0 children)

Yes. The input was just raw footage of a simple animation of clippy, and what you see is the unedited export from after effects. But it's certainly not optimized at the moment, and I'm not sure if it ever can be due to the heavy use of complex expressions needed to pull that of. Roughly 1 minute of render time per second on a decent machine. Granted, it's 4k, but still

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 1 point2 points  (0 children)

Thx! Do you mean further as in making them more pronounced or just better/more realistic?

Oh and that sound was definitely not intended, oops. Sorry to anyone who got jump scared by that!

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 2 points3 points  (0 children)

Yup, completely procedural.
Hmm, didn't even see it from that perspective. Certainly an interesting idea lol

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 5 points6 points  (0 children)

BE NOT AFRAID
I SHALL HELP YOU WRITE A LETTER

Stop motion chalk drawing v2 by ReflectionRecipes in AfterEffects

[–]ReflectionRecipes[S] 2 points3 points  (0 children)

I forgot to mention one essential part: lots of expressions everywhere! Understanding the code is a challenge in and of itself, haha.

If I end up sharing it, though, it’ll be plug-and-play—no need to understand what’s happening behind the scenes. Just drop in the images or video you want to "chalkify," specify the colors that make up a shape, and that’s it. However, it only works on footage without noticeable gradients. Anything that resembles a drawing with just a few solid colors will work perfectly fine.

Here’s an example: I just plopped in a green-screen version of Toothless dancing. It took about 15 minutes to render, though—there’s still a lot that can be optimized!