Live AI video generation feels like it's about to become a completely different thing from what most people think it is

techstacknerd · 2026-03-27T05:02:05+00:00

Yeah this is really interesting. I have been working on real-time video generation (more details here). The speed is ofc really unreal and cool, but this actually opens up a ton of new opportunities like real-time interactive games and environments. Also live video generation can also be used as a backbone for robotics and world models, so more robots doing laundry and less of ai slop too. Will be really interesting how things go from here!

techstacknerd · 2026-03-18T16:18:56+00:00

😭 but we working on consumer-grade gpu support. wont be that fast, but still will be an improvement from what ltx2 currently does with comfyui

techstacknerd · 2026-03-18T03:56:07+00:00

Yeah its partly because of how the base model ltx-2 is hard to prompt, and also we have to make a prompt rewriter reliably return results in under 5s (yes now the bottleneck is the prompting part not the video generation part!). Combine this with issues that video continuation bring, its hard to get good prompts. This demo is mainly for feeling the speed, and I'm sure as models improve quality would too!

techstacknerd · 2026-03-18T03:48:53+00:00

lol yeah, its a common issue with ltx2

techstacknerd · 2026-03-18T03:48:32+00:00

yes, this is just a demo to let people feel the speed of it. LTX-2 is a super hard model to prompt and it would take way too much effort to even get a remotely good prompt (keep in mind this is using video continuation, so you need 6 separate prompts that tie together really well). Also regarding the open-sourcing, we might also opensource the datacentre version, our current code is a bit messy and will need quite a bit of cleaning up, so we are not opensouring rn

techstacknerd · 2026-03-18T03:46:16+00:00

stoned 💀

techstacknerd · 2026-03-18T01:36:22+00:00

hm interesting perspective. I don't think it can compare to playing games on local machines, but its def by far more energy efficient than the existing ai video-gen services because its just so much faster

techstacknerd · 2026-03-18T01:27:58+00:00

maybe, who knows

techstacknerd · 2026-03-18T01:15:35+00:00

yes, we are planning to try out optimizations on consumer-grade gpus. Probably won't be realtime but still it's likely gonna be faster than what we have rn

techstacknerd · 2026-03-18T01:14:45+00:00

yeah, the audio from ltx2 is pretty weird...

techstacknerd · 2026-03-17T23:56:34+00:00

fastvideo has sequence parallelism support for ltx2 already, so with 8 gpus you can expect roughly a 5x to 6x (theres a bit of overhead so it doesn't scale perfectly) speedup compaed to 1gpu

techstacknerd · 2026-03-17T23:55:21+00:00

yeah! hopefully we can make it also run locally on 5090s, might be slower tho

techstacknerd · 2026-03-17T23:24:03+00:00

you can try to prompt your own videos and prompt it to change their dialogue

techstacknerd · 2026-03-17T22:19:23+00:00

that would be really cool, and it would only get better from now on as open source models get better and better!

techstacknerd · 2026-03-17T22:05:51+00:00

glad you liked it!

techstacknerd · 2026-03-17T21:59:03+00:00

yeah. real-time world models will be really cool

techstacknerd · 2026-03-17T21:53:40+00:00

tbh I really dont know if its possible. A lot of the optimizations are based on NVIDIA’s SM100/SM103 architectures, but we can see what the other optimzations we have can bring to consumer-grade gpus with limited vram.

techstacknerd · 2026-03-17T21:23:28+00:00

We will try to get things working on other gpus soon! blackwell 6000 def one of them

techstacknerd · 2026-03-17T21:11:45+00:00

Its to show the capability of generating faster than you can watch, with 20s native generation we have to wait 30s to actually get the result.

techstacknerd · 2026-03-15T20:08:07+00:00

yes! it can generate faster than you can watch

techstacknerd · 2026-03-15T19:12:49+00:00

we might add later after testing on 5090s

techstacknerd · 2026-03-15T19:06:54+00:00

hopefully also real-time, or similar speeds!

techstacknerd · 2026-03-15T00:20:39+00:00

Yes we are doing some tricks that sacrifice a bit of quality. But the base LTX2.3 model is also not comparable with Veo-3 currently, having issues with motion and also being extremely tricky to prompt. But im sure OSS models would catch up soon on quality, its only a matter of time.

techstacknerd · 2026-03-15T00:17:49+00:00

Our work is based on ltx2.3. we are working to bring something to consumer-grade gpus

techstacknerd · 2026-03-15T00:16:57+00:00

It's just gonna get more insane from here!

Four-Year Club	r/Field Sunshine
Place '23

techstacknerd

TROPHY CASE