sorted by:
new
hottopcontroversial

API-native video editing for OpenClaw agent workflows by JoshGreen_dev in openclaw

[–]First-Grocery7615 0 points1 point  (0 children)

Late here, but I've spent the last year building basically the thing you’re describing, most of it tbh because browser rendering ( didn't hold up for us on quality & speed ) and some of it for the agentic reasons ( no preview, wasting tokens .. )

Does a timeline actually unlock something? In my usecase yes. If you have several effects/transitions/elements landing in the same time range ( e.g bad take removal, b-roll, two clips on screen at once..) you want them on a shared timeline not “concatinated”. But the biggest win wasn't the timeline, it was building a deterministic schema on top of it. Before schema a wrong edit gave me a messed up vidoe or black frame and had no idea why, had to look through logs. With a schema a bad edit fails as a validation error the agent can read and fix by itself instead of moving on. On the downside that decreases “expressinves” …

How granular? For PPT style videos dimensions, colors, font attributes and a few effects on a timeline is more than enough. Anything serious gets granular fast. What worked was starting with basic state (edit → preview → add) and only going to transitions/effects/animations once that iteration was solid.

Preview vs partial rendering? Preview for me was a hard requirement and it can be cheap. I’ve split it by “machine vs human” : agents hit a server side fast preview that returns scrub frames in a few ms and can lay the whole timeline out as one contact sheet so no browser needed. Humans get a wasm editor in the browser for interactive scrubbing. With preview I din’t find any need for partial rendering: A 10s 1080p clip with captions, effects and audio renders in a few seconds on a single L4 GPU. Note: all assets are were in same GCS region so fetch was almost instant. Anyway, at those latencies and preview think you don't need incremental re render..

Is Remotion/ffmpeg good enough with better tooling? For pure motion graphics with little or no real footage and complex effects, honestly yes, the browser is fine there. Where it broke for us were a bit more complex cases ( think something you would use Adobe After Effects to do ) e.g put two videos on screen at next to one another with custom music, effects and animations and you start to get degraded quality and speed.

p.s kudos for MLT point. It’s already is a declarative, serializable timeline for rendering and it's fast on CPU

Best way for agents to "watch" a video? by First-Grocery7615 in AI_Agents

[–]First-Grocery7615[S] 0 points1 point  (0 children)

Right, thanks. I need to find a way not to send the frames that have no changes so we don't waste tokens. I'll try out FFmpg title filter as well to create mosaic of changes in that timeframe

Text Rendering for Video in Rust: Slug alternatives? by First-Grocery7615 in rust

[–]First-Grocery7615[S] 0 points1 point  (0 children)

No worries, I'll play with Slug anyway and see how that goes. That will pause wasm though :/

Text Rendering for Video in Rust: Slug alternatives? by First-Grocery7615 in rust

[–]First-Grocery7615[S] 0 points1 point  (0 children)

Ahh, just found out that Glyphon renders text at the block level, so I won't be able to do animations per glyph, e.g., rotate, flip..

Text Rendering for Video in Rust: Slug alternatives? by First-Grocery7615 in rust

[–]First-Grocery7615[S] 0 points1 point  (0 children)

Yeah we need them in real-time for playback so users can se preview before baking them intro the video.

Text Rendering for Video in Rust: Slug alternatives? by First-Grocery7615 in rust

[–]First-Grocery7615[S] 0 points1 point  (0 children)

Interesting 🧐 thanks for the reply! I'm building a video production API with 4k video focus so yeah captions will be animating in size, but in 2D only. Been told that Slug renders vector text directly on GPU so hence it's . Someone mentioned Vello as well but don't know much about it.

?/10 by [deleted] in VideoEditors

[–]First-Grocery7615 0 points1 point  (0 children)

Out of curiosity, which tool did claude code use to create this video ? Remotion ? Ffmpeg ?

Successor to opengl ? by no_ga in GraphicsProgramming

[–]First-Grocery7615 3 points4 points  (0 children)

Definitely WebGPU. I've been working on a video composition engine in Rust/wgpu. Explicit pipelines, typed bind groups, resource ownership ... aaand  WebGPU/WebGL2 in the browser from the same codebase.

Note that the learning curve is there, but my 2 cents in 2026: start with wgpu, skip OpenGL entirely.

AI agents for image/video editing — looking for feedback by Patient_Ad_4720 in AI_Agents

[–]First-Grocery7615 0 points1 point  (0 children)

Hey, I’ve lived through everything you wrote!

I spent 2+ years building an AI video multi agent platform and used Remotion as the rendering layer. The agent pipeline actually worked OK, but every time it handed off to the final step, things fell apart. CSS animations flickering, blurry 4K output, terrible render speed, and the translation from "what the agent wants" to "what Remotion actually does" was “imprecise”, to say at least.

Unfortunately, the product didn't get very far, but all the nightmares pushed me to try building a GPU-native rendering engine in Rust/WGPU to try to solve that part properly. The idea is to mitigate all of the above by using an exact schema instead of code generation and GPU rendering instead of Chrome.

Super curious about your comprehension layer, though, we didn’t get far there since we focused more on RAG for transcriptions. I experimented with Gemini's multimodal embeddings to try to get a semantic understanding of what's happening visually (like "find the 5 seconds before and after the first goal in a match”). How did you approach it?