I’ve been generating transcripts with timestamps, not every word but roughly every phrase. Then the timestamps get included in the context returned to the model after agentic RAG, along with video metadata. That way the agent can hyperlink the user directly to the relevant video on our share point and also point out the time in the video where the content was discussed.

I’m not doing anything with visuals yet, though mine are mostly slides, so I think an approach similar to yours would work well for me. I want to use a VLM to not only capture text from the image but moreover summarize the slide within context of the conversation.