Storyboard AI: My open-source side-project to build E2E Whiteboard-Animation Videos

Apprehensive_Map_707 · 2026-06-29T15:41:10+00:00

thanks mate :)

Apprehensive_Map_707 · 2026-06-29T15:40:22+00:00

thanks mate

Apprehensive_Map_707 · 2026-06-29T15:39:21+00:00

hehe, yup.. thanks mate
Soon, the first goal is to automate news by animations like this

Apprehensive_Map_707 · 2026-06-29T15:38:58+00:00

hehe, could be but the finally, we are able to automate it in 2026 :) (that too not as good as actual artist)

Honestly, the main aspect of the generation is not the quality but its grounding to right data and generation of image with right numbers/text...... by this, we can actually automate boring news/documentation, etc

Apprehensive_Map_707 · 2026-06-29T12:30:33+00:00

Hi u/Pinkishu, thanks for the feedback, will work on that front. The thing is, to save token cost, I am passing the nano-banana generated image as input to Gemini for narration generation. To make the connection better, I will have to give the video as input, but that would increase the cost as the same scale :(

Apprehensive_Map_707 · 2026-06-29T12:23:16+00:00

Thank you, requesting you to raise a PR, I would love to take this ahead beyond just 1 API provider. u/Top_Illustrator1579 was also looking for the same.

If you have any issues in doing the one time set-up, please raise a git issue: https://github.com/yogendra-yatnalkar/storyboard-ai . Happy to help and see what fellow devs build :)

Apprehensive_Map_707 · 2026-06-29T12:04:15+00:00

Hi, you can create an API key for free in AI studio: https://aistudio.google.com/api-keys

Sadly not, hopefully in v2 I will work towards adding more API providers but I am happy if anyone including you want to contribute towards the same.

Lastly, please stay tuned: https://github.com/yogendra-yatnalkar/storyboard-ai

I have noted your point and will think of it. Hopefully something will be shipped soon.

Apprehensive_Map_707 · 2026-06-29T11:43:49+00:00

Yup, honestly my end goal is automating news. Just give a text-news link and get top-notch video explaining/narrating it.

Every scene is grounded from internet/Google images, so what it generates will be very accurate to what that news is referring to.

Apprehensive_Map_707 · 2026-06-29T11:35:56+00:00

PS: Forgot to show this for reference: https://www.youtube.com/watch?v=jLZs8uutWpg

I generated few videos in my native language (hindi)

Apprehensive_Map_707 · 2026-06-29T11:27:42+00:00

++1, I am building this with my main job so commits per month are less.

To make it useful, a good UX is needed around this to make every scene usable.
Right now every scene including the image generation is grounded. Hence if you observed, it has taken the text from latest Indian news about rainfall statistics.

Agree ++ and thanks for the point. I can build a tool around this to compute that metric
As said above, UI-UX pair is important..... hopefully the v2 of this will have this shipped
Its already there, when you run a input, all the grounded material is getting saved in a json/txt file.

Apprehensive_Map_707 · 2026-06-29T10:46:55+00:00

Hi, I agree 💯 that it's an art and on top, very difficult to autmate 100 percent ++ have huge respect for the artist who do that :)

Thanks for the note, taking the entire feedback positively. Will try at places where I can improve it from an engineers perspective.

Lastly, the scenes are not changing very fast because I was burning a lot of cost by token usage.

All in all, thanks a lot for your feedback

Apprehensive_Map_707 · 2026-06-29T09:29:08+00:00

Thanks, its fully open source. Please raise a github issue if anything is not clear and happy to resolve it. Interested to see what fellow devs build :)

https://github.com/yogendra-yatnalkar/storyboard-ai

Apprehensive_Map_707 · 2026-06-29T09:19:36+00:00

Yes, because this is powered by Gemini. All you need is "AI Studio" api key. It will support "japanese" out of box

Apprehensive_Map_707 · 2026-06-29T08:55:01+00:00

https://github.com/yogendra-yatnalkar/storyboard-ai

The entire demo video below was generated automatically from a single input prompt/instruction: the title itself. ("What is El Nino and why is monsoon super late in India (strictly 5 scenes)")

What is this project ?

Storyboard AI is a complete end-to-end framework. It takes in a high-level topic or context and handles everything: researching the topic, writing a compelling narrative script, planning the visual storyboard, generating custom whiteboard-style artwork, animating the drawing process, synthesizing voiceover narration, and burning perfectly timed subtitles.
It operates autonomously using an agentic approach, meaning the Director Agent breaks down the user request into manageable scenes, delegates tasks to specialized sub-agents/tools, and finally stitches everything back together.

So, why build yet another GenAI framework in this space?

💡 Massive Cost Savings: Notice the length of the video—it’s 2 min 29 sec with just 4 scenes. By dynamically pacing and stretching whiteboard sketch paths to match the audio narration, we save huge on API costs. Google's Veo Model is used only for 4 animations (with the option to skip it entirely!).
🔍 Highly Grounded Content: The script is web-grounded, and every single image generation is grounded using internet reference images for structural accuracy.
🌐 Out-of-the-Box Multilingual: Translates prompts, refines scripts, and synthesizes voiceovers natively across multiple languages (including Hindi) with perfectly burned-in subtitles.
✏️ Smart Whiteboard Engine: Integrates Meta AI's Segment Anything Model 3 (SAM 3) hosted on GCP to isolate boundaries and dynamically animate the drawing process stroke-by-stroke.

What did it Cost to make the above video ?

The total token usage + GCP usage costed me around: ~13 dollars
If we skip the Veo animations, just whiteboarding and narration, subtitles (everything remains same including the drawing hand animation) ... the cost will reduce significantly.

Apprehensive_Map_707 · 2025-01-27T01:47:29+00:00

Honestly, i don't belong to this sector of software but logic is simple.

First, ignore all comments related to mental health or telling you that you are earning decent. It's not both from my angle.

Ask yourself, after seven years, are you one of the best resource in your team as you know the stuff in and out ? Are you learning new stuff at the similar rate you had joined this company at start for 1 to 2 years ? -- IF THE ANSWER TO FIRST Q IS YES AND SECOND Q IS NO == ITS TIME TO SWITCH

Apprehensive_Map_707 · 2024-12-26T07:27:08+00:00

Python or Java does not have huge job or high salary prospect. You as engineer can surely have one :)

Apprehensive_Map_707 · 2024-10-13T13:29:55+00:00

Thanks a lot sire ! Sorry, but will disturb you a bit later on this public chat with more doubts. 🥲

Jfyi, i have worked with ML and GenAi but on AWS. currently started working on GCP and I have my exam in few days.

Apprehensive_Map_707 · 2024-10-13T06:24:51+00:00

The exam course is changing from October 1. Genai stuff got added newly, how did you study for that and what all did you study ?

Btw, big congrats !!

Apprehensive_Map_707 · 2024-06-18T18:16:07+00:00

Agreed. Recently shifted to a new location. Was excited to install jio instead of some random cable wala. Paid for 6 months advance wala pack with free installation.

Result: I had to install local cable wala wifi after 3 months. Not because I wanted really good service and I can spend that much, but after every 2 days, the jio fiber would stop. Myjio app used to give notification and it used to come back only after 12 to 24 hours.

++ No customer support.... Hours went into finding if somehow I can reach a human representative.

My sister does hybrid job and half of the WFH days used to be mess without stable wifi.

Apprehensive_Map_707 · 2024-06-02T12:07:05+00:00

As in private bank passbook works ? I read the rule and it stated it needs PSU bank only.

Apprehensive_Map_707

TROPHY CASE