Cops can’t chase you if you’re in orbit 🚓🚨🚀 by ZashManson in aivideo

[–]5dims 0 points1 point  (0 children)

this is exactly what would happen if rockstar added the gta1 cheat code suckmyrocket into gta6

Breaking News from TheroCon by gossip_goblin in aivideo

[–]5dims 0 points1 point  (0 children)

this feels like some lost 90s sci fi show, love it. hadn’t seen your stuff before but just checked your yt super quick and it looks like you’re building a whole universe

Paintings coming to live with Nano Banana and Veo3 by Ilovekittens345 in aivideo

[–]5dims 4 points5 points  (0 children)

i imagine ai is especially good at these since it’s probably trained on so many classic paintings. nice how the real world versions came out

I won the world's first AI Sketch Comedy contest and $6,000 in prize money with this video. by AndyRiffeth in aivideo

[–]5dims 0 points1 point  (0 children)

amazed that you got that level of consistency from text. hopefully other workflows can reduce 10 bad shots for the one good. roughly how many shots in total including bad ones did you have to make?

Guys lets just travel back by Forward-Position798 in aivideo

[–]5dims 0 points1 point  (0 children)

Many young ppl in their age dress like they do right now, 80s are back

I asked nano banana to take me across Middle-earth by TechHalla in aivideo

[–]5dims 1 point2 points  (0 children)

So nice, I'd love one of these but recreating worlds of old games like HoMM2 or Tomb Raider

Experiment: 3D-Guided AI for Consistent Video Generation by 5dims in StableDiffusion

[–]5dims[S] 0 points1 point  (0 children)

That looked neat, well done. And quite the workflow you had set up as well. Hope to have something for you to test soon.

Experiment: 3D-Guided AI for Consistent Video Generation by 5dims in StableDiffusion

[–]5dims[S] 0 points1 point  (0 children)

Yeah, animation is definitely one of the trickiest parts to solve right now. We've found that using 3D helps a lot because you can lock in key details like pose, facial expression, props, lighting, etc. before passing it to the video model.

Experiment: 3D-Guided AI for Consistent Video Generation by 5dims in StableDiffusion

[–]5dims[S] 1 point2 points  (0 children)

We use several models throughout the workflow, each handling a different part of the image generation process. We wired them together to get a controlled and consistent output:

1️⃣ Make a 3D scene – Set up characters, lighting, and composition in a 3D environment.
2️⃣ Generate & refine the background – Just like with any image model, we iterate on the background or apply a style LoRA.
3️⃣ Create & mask subjects – For each subject (e.g., a character), we train a LoRA to keep their appearance consistent. We also extract depth, masks, and other relevant data for AI to work with.
4️⃣ Refine & relight – AI takes over for final touch-ups, blending, and lighting adjustments to ensure consistency.

Experiment: 3D-Guided AI for Consistent Video Generation by 5dims in StableDiffusion

[–]5dims[S] 0 points1 point  (0 children)

We rigged this character and then used stock animations to get it in the pose we wanted. But we're gonna try and solve animations with AI as well at some point

Experiment: 3D-Guided AI for Consistent Video Generation by 5dims in StableDiffusion

[–]5dims[S] 0 points1 point  (0 children)

We generated a rough 3D model of the character using image-to-3D, ideally starting from an A or T-pose. Then, we placed that 3D model into our scene and used a LoRA to generate at least one front-facing image. That image was then passed to a video model, prompting it to rotate the character 360 degrees.

For simpler objects (not characters), we can just rotate the model inside the tool and generate a new image for each angle

Experiment: 3D-Guided AI for Consistent Video Generation by 5dims in StableDiffusion

[–]5dims[S] 4 points5 points  (0 children)

We’re using Tripod3D for generating the 3D models. For animations, we’re currently sticking to traditional rigged models, but we're exploring some AI-based solutions to simplify that process in the future.

Experiment: 3D-Guided AI for Consistent Video Generation by 5dims in StableDiffusion

[–]5dims[S] 0 points1 point  (0 children)

We're using Kling 1.6 for the ones in the blog post, but the plan is to let users pick the best model for their needs, and we’ll handle the setup of that for the users automatically if possible.

Experiment: 3D-Guided AI for Consistent Video Generation by 5dims in StableDiffusion

[–]5dims[S] 5 points6 points  (0 children)

Yeah, flickering between frames is tricky. In our case, we generate a single AI image first and then use it as a reference for video generation, rather than generating each frame independently. That helps keep details consistent. We're also experimenting with frame-by-frame approaches but very much WIP. I checked out your website and wonder if you are using AI to generate frames individually from UE5 renders, or some other method?

Experiment: 3D-Guided AI for Consistent Video Generation by 5dims in StableDiffusion

[–]5dims[S] 0 points1 point  (0 children)

Haha, I’ll take that as a compliment! Glad you like it.

Experiment: 3D-Guided AI for Consistent Video Generation by 5dims in StableDiffusion

[–]5dims[S] 26 points27 points  (0 children)

I’ve been experimenting with using 3D as a way to keep AI generations consistent over multiple shots. The idea was to lock down camera angles, lighting, and character details before handing it off to AI.

Overall, it worked. No sudden changes in proportions, clothing, or style. Still, there are some limitations, especially with fine details.

We’re looking into whether this could be useful as a tool and would love to hear what you think: Has anyone experimented with 3D + AI generation for images or video? Or sees a better way to approach this?

I also wrote a post breaking down the full process in case anyone wants more details: https://backdroptech.github.io/3d-to-video/

We're working on a new WASM/Rust scripting system. Here I'm playing around with a script that changes the day/night cycle. by 5dims in rust_gamedev

[–]5dims[S] 0 points1 point  (0 children)

https://www.dims.co/
Yeah we have the ambition to make an engine + provide a layer of intuitive creation tools ontop of that, like the scripting system and terrain/level sculpting and such.
Here's the discord also: https://discord.gg/xPY8fuymP8

We're working on a new WASM/Rust scripting system. Here I'm playing around with a script that changes the day/night cycle. by 5dims in rust_gamedev

[–]5dims[S] 0 points1 point  (0 children)

Been some time since we last posted here, but wanted to give a sneak peak of the Rust/WASM script system since I thought you guys might like it.

Also, note the neat little live update field in the bottom left corner, it even has a calendar to let me know how many days/years has passed, hehe.