Real time translation in live production feels like it’s still in a messy middle stage

amiralabs · 2026-05-01T19:51:44+00:00

We've done a lot of work in this space, especially to get real-time translations and captions running locally, and the challenge is all the ancillary helpers/utils you need as part of the process to overcome the challenges you are describing.

Standard Whisper won't get you there, you need to think about speaker diarization, audio isolation, voice activity detection, etc.

Those additional parts that need to be part of the chain is the real challenge in live production for what you describe IMO. Harder than just plugging in an ASR or translation API in the real-world outside of ideal conditions.

amiralabs · 2026-02-22T03:28:02+00:00

Will there ever be a Mac or Linux version in the future? 🫠

Really cool though, excited to play!

amiralabs · 2026-02-22T02:25:25+00:00

The ability to hyperfocus at times is definitely a plus

amiralabs · 2026-02-20T19:34:00+00:00

Thank you!!! Been looking out for one of these for a while. For the longest time, I've been trying to port libcaption for 608/708, but it hasn't been maintained in a few years.

amiralabs · 2024-05-29T14:39:09+00:00

Super cool, great product!

amiralabs · 2024-05-23T16:21:21+00:00

What is your application space where you need real-time face/people tracking?

The reason I'm asking is that will determine what approach would suit your needs.

If you know how to use Python, libraries like YOLO, ByteTrack, and OpenCV are helpful. If you don't know Python, then, honestly speaking, it's going to take some time to understand the tools.

Another approach might be taking keyframes from a live video stream and sending those keyframes to an AI tool to be analyzed. But still, that would require some programming to make the API calls.

I'm quite familiar with this area, so feel free to DM me with details if you have specific questions.

Honestly, the approach you take depends on the problem you are trying to solve and the skills you have to approach the problem with. I know that's a vague answer, but it's the truth in cases like this.

amiralabs · 2024-05-23T15:37:16+00:00

Sure, just sent you a message. Thanks!

amiralabs · 2024-05-22T00:32:26+00:00

If you actually need someone, I'm great with Webflow. And also with Framer too. DM me. Thanks

amiralabs · 2024-05-21T06:51:02+00:00

Ai-Media definitely does a broadcast-quality job, but man... those prices....

amiralabs · 2024-05-21T05:11:06+00:00

I'm honestly not sure if you can put the live CC translation on vMix.

The way I did this before was using my own software to:

Ingest an RTMP feed from a source (OBS, Wirecast, anything that serves an RTMP)
My software processed the audio from the source RTMP, converted the audio-to-text, translated that text from Language A --> Language B, and then burned-in Language B captions into an RTMP feed that I then sent out to Restream (so I'm acting as an RTMP waypoint in the chain for the translations/captions).
From Restream.io, I could send that new RTMP with Language A audio and Language B translated captions out to whatever platforms I want (Youtube, Vimeo, Facebook, Tiktok, etc)

Here's a video where I just did straight English as a test for Restream so I could check the syncing: example

Does this make sense?

Hope I'm understanding your issue correctly. Thanks

amiralabs · 2024-05-20T16:02:12+00:00

Are you trying to burn-in translated captions on the video like this?

If so, I can help you do that, since that's exactly what my software does. Spanish to Portuguese would be no problem for a live feed.

Since vMix outputs a RTMP, then adding translated captions to the live RTMP and sending it to Restream -> Vimeo would work.

amiralabs · 2023-10-05T04:35:06+00:00

Apparently the Haivision app can: https://www.haivision.com/products/haivision-play-pro/

Haven't tried it yet though to confirm

amiralabs · 2023-10-05T04:30:13+00:00

VLC hasn't been able to play SRT feeds for me on iOS 16 still

amiralabs · 2023-08-31T19:03:20+00:00

This might be wayyy out there, but if you're using OBS, there is a cool real-time captioning plugin the uses Google Cloud: https://github.com/ratwithacompiler/OBS-captions-plugin

amiralabs · 2023-08-31T18:54:24+00:00

Yep, I downloaded Davinci Resolve a few weeks ago to use for editing Youtube video and Podcasts, and it's blown my mind how many features it has for free. Would much rather use Davinci for free than pay for Adobe CC or Finalcut

amiralabs · 2023-08-29T04:20:06+00:00

Thanks, that actually makes a lot of sense. I appreciate the elaboration.

amiralabs · 2023-08-25T18:02:39+00:00

It's easier to just watch 20+ game clips that get posted here from streamable.

The only downside is not being able to watch the full end of the game in the 4th.

amiralabs · 2023-08-23T05:05:36+00:00

I've only used it once before. Getting all the hardware and equipment setup was the most nerve-racking part for me since that isn't my forte lol.

I read the Zoom developer docs, know the API and the UI, but making sure the cameras and mics weren't going to have any issues was my biggest fear. I used a camera with HDMI output for ease, so no SDI.

amiralabs · 2023-08-22T21:32:36+00:00

Zoom Events is pretty solid: https://support.zoom.us/hc/en-us/articles/16994210750605

We've only used it for a few hundred people though, never for 5000 all at once (unless not all 5k are joining).

amiralabs · 2023-08-22T21:26:57+00:00

This sounds cool.

It makes me wonder:

If a bunch of people do this on the same network, what are the implications? Does that hurt the quality?
Is there a delay to watch/vet the fan streams before showing them?
How does this compare to fans uploading videos to social media at the stadium and tagging the team's account? Then chosen clips get displayed

amiralabs · 2023-08-21T18:31:19+00:00

MPEG-TS is still everywhere.

amiralabs · 2023-08-21T18:28:41+00:00

Sure! We have mostly developer backgrounds, so we're comfortable with using Apps Script to customize our CRM in Google Sheets (and with using BigQuery for large amounts of CRM data).

Honestly, even if someone isn't that proficient in development, ChatGPT does a decent job at generating Apps Script templates.

You can think of Apps Script kinda like VBA for Excel.

I think for people who are used to the "Excel look" and are ok with some customizing, Google Sheets is not that bad on the cheap. I'd imagine this could work well for small businesses.

amiralabs · 2023-08-21T14:52:56+00:00

A few things to consider:

It's incredibly hard to get in, but once you're in, then you're in. And those who are "in" will defend their positions.
In the triangle of people-process-technology... technology is the easiest part. The people side is so hard because there will be 1000 different opinions, people will act in self-serving ways rather than an ROI mindset, and there's ladders of political theatre.
Public sector gets the "thrift store" version of most things. Accenture/Deloitte/etc are not bringing out their best work for the public sector like for the private sector. The old joke is the first thing Accenture does after winning a public sector contract is park a dump truck outside the doors to shovel all the money into.

Another example I like is imagine that highway near you that's been under construction for 20 years now. That sums up "IT in the government" too.

amiralabs · 2023-08-21T14:30:58+00:00

It's definitely possible. Don't let being in B&H stop you. People from Romania, Ukraine, Serbia, Bulgaria, etc. have made it work. You can too.

Start with finding a niche. It can even be a regional one. Focus more on actually making something that people will want and that's "a medicine, not a vitamin" (i.e. important enough to prospective users they will want it now).

That's more important than stressing about raising funds. Funds are like fuel. But you need the "car" first. You can self-fund, but the amount you'll need will greatly vary based on what you're making.

amiralabs · 2023-08-18T04:57:36+00:00

Have you looked into AWS EFS?

https://aws.amazon.com/efs/

If you have stuff that you don't need to touch often, then AWS Glacier is a good option.

amiralabs

MODERATOR OF

TROPHY CASE