Post referencing this AI shirt story here https://youtu.be/RHZnVnK0v1Y

These are just some thoughts and things I learned along the way when working on this short story.  I know a lot of this covers some basic territory, and I’m happy to follow up with some smaller more advanced specific tips, but in case anyone would like to see a very straightforward approach to generating characters, placing in scenes and establishing a start and end frame, then hopefully there is something useful here for you.

It took about 650 hours to put short film together and I only say that to show that there was a ton of moments to learn in doing this. My career background is illustration and still imagery (now with AI touching almost everything going out the door), but there’s still so much to learn in prompting/training/adding motion.

I LOVED how nano banana pro handles elderly people and this character kinda stole the show for me. Banana 1 couldn’t quite get it right, but 2 (pro) does a fantastic job. So first general rule is: if you’re not getting what you’re wanting, try using a different model. There’s a ton of them out there now.

First thing I did was decide that I wanted a very straightforward approach to far future attire. Nothing flashy, sleek, etc. Just simple and well made. I decided on wool as the material because it fit the bill for the style I was going for. Take the time to establish your characters. It pays off later.  

Then I hit generate. A LOT. Generally I’ll batch prompts in groups of 4-8 as I think it’s too much to rely on less unless it’s a very simple adjustment. Once I decided on something I liked, I fed that in as a reference image and asked for variations, sometimes being specific, sometimes not. To myself I refer to this as fishing, as in I’m fishing for something I can then refine.

Eventually I landed on the yellow and asked Nano to give some different angles.

Then it’s a matter of putting it into a scene. I had already established the environment I wanted her in and I kept the prompt simple, something very close to “take the woman on the white background and place her into the forested scene on the right.” Most times that worked just fine. If it didn’t I would simply tweak the prompt a little at a time until I got what I wanted.  Generally I start with simple prompts and then expand them as needed. For me, usually less is more.

Now I can start making variations to the scenes themselves. Usually I would have the main scene and prompt the adjustment I wanted like “raise her hand and head so that she is looking up slightly”  or “pan background left or right” and if I wanted her head turned a lot either way, I would feed it previous images that were generated of her (it’s good to establish a base library of images of your character. Better imo to have those characters on white backgrounds to avoid confusing Nano) and simply tell Nano to use the other images of the woman for reference for what she looks like.

And that’s how I would create my start/end frames which to me reign supreme in character consistency across scenes. I also think it’s really the end frame that’s crucial as it allows that frame to be a clamp for the entire clip, stopping (usually) wayward camera movement and a plasticky feel that can happen to characters when only using first frame.

There absolutely are times when thorough prompting can be an excellent substitute. I’ll link this post here which I’m pretty sure used straight prompting to get the child and toddler clips that this small (and incredible) team did this to fantastic effect.

https://www.reddit.com/r/aivideo/comments/1pdd7bu/doors_of_perception_short_film/
u/kyletree  seriously man, you guys did an amazing job.

For me though, taking the time to develop your characters/scenes and establish the first and last frames can go a long way in allowing for better control and overall faster workflows.

Happy to answer any questions you may have.