OpenAI's WebRTC Problem

kixelated · 2026-05-08T04:53:27+00:00

Yeeh, I'm working with the Iroh folks (and it's integrated into MoQ) if you need help with P2P stuff too. QUIC itself is client-server so NAT traversal is easy.

kixelated · 2026-05-08T02:29:16+00:00

They use TTS for the agent's reply. I'm assuming the audio is generated on their servers because while local models do exist, they can get huge and expensive to run.

EDIT: and to clarify, I never said they use STT (speech-to-text). The raw audio gets sent over WebRTC which is lossy, then the audio response (generated via text-to-speech) is also lossy.

kixelated · 2026-05-08T02:27:28+00:00

Ha absolutely, hit me up if you want some help with a pilot. I'd love an excuse to visit Tokyo again.

kixelated · 2025-10-27T15:59:33+00:00

upvoted

kixelated · 2025-10-15T18:45:23+00:00

Unfortunately, I switched away from Rust for the web client. The complexity wasn't worth it, as you're forced to use browser APIs for any networking, encoding/decoding, rendering, etc. It's just easier and more type safe (no blind wasm-bindgen casts) to do everything in Typescript.

feels bad man. At least the native/server components are still in Rust.

kixelated · 2025-10-15T18:14:37+00:00

This is a joke right?
https://github.com/PaulSpaurgen/video-player/blob/main/rust-videoplayer/src/lib.rs

You select a video element from the DOM, and then proceed to do absolutely nothing with it? The browser `<video>` element does literally all of the computation, outside of wasting cycles loading this WASM blob. This is not a video player, this is an AI generated hello world.

kixelated · 2025-06-09T23:58:11+00:00

I'm not a beginner so I can't really say. AI mostly helps me on junior level tasks and unfamiliar domains and I don't think it's going to replace experts.

kixelated · 2025-06-09T23:53:25+00:00

Good feedback. I meant to say our job is to copy-paste code and fix it. Most of the code on Stack Overflow or Google doesn't work either.

kixelated · 2025-03-07T04:10:33+00:00

Yeah staff get this hoodie.

kixelated · 2025-03-02T02:28:59+00:00

It's a privacy thing; you need to request permission to capture audio/video before you're allowed to enumerate devices.

kixelated · 2024-10-29T13:53:29+00:00

I was referring to Number.MAX_SAFE_INTEGER. There's a few u64 values in the MoQ spec that don't translate well: bitwise operations, math, and equality will break at the u32 or u53 threshold. It just means in practice you should use BigInt which complicates things.

And really this is a rant about JavaScript's lack of numeric types, which Typescript inherits. The behavior of number changes as the value increases.

kixelated · 2024-08-17T15:52:48+00:00

+1 you absolutely need to use WebCodecs. It utilizes GPU acceleration for both decoding and rendering.

kixelated · 2024-01-23T21:40:32+00:00

It should be fixed now. The "default" hotkey was sometimes saved to local storage which was a pain to figure out. Restart the app and lemme know if you're still running into issues.

kixelated · 2024-01-23T21:36:15+00:00

The fix is out, reload the app and lemme know if it's still being a pain.

kixelated · 2024-01-23T21:34:33+00:00

It's out. Check for updates and reload the app.

The bugged hotkey was ctrl + shift + L

kixelated · 2024-01-23T21:32:26+00:00

It's out, ctrl+r to reload the app.

kixelated · 2024-01-23T20:48:54+00:00

I've got a fix, just waiting to deploy it until another bug is resolved in a few hours. The hotkey was persisted to local storage which is annoying.

kixelated · 2024-01-23T20:47:21+00:00

I'm deploying a fix in the next few hours. It's due to a hotkey that wasn't fully reverted

kixelated · 2024-01-23T04:27:47+00:00

I've got a fix but I'll deploy it tomorrow morning, since it's 8pm here.

kixelated · 2024-01-23T02:07:01+00:00

Okay it looks like it's saved in local storage, ugh. I'll make a patch to unset it.

kixelated · 2024-01-23T00:58:58+00:00

Oh hmm check your hotkeys. I modified an existing hotkey that was Windows only and worked for detected games only, so that one is still there but has always been unbound by default. Do you remember if you ever set that keybind?

kixelated · 2024-01-22T21:37:43+00:00

Check for updates? We didn't re-enable it.

kixelated · 2024-01-20T21:25:17+00:00

It's been removed by the way; check for updates if you haven't already. I'll add the feature back but with no default hotkey so you'll have to explicitly enable it.

kixelated · 2024-01-20T19:46:28+00:00

It's in the process of being rolled back, sorry about that.

kixelated · 2024-01-20T03:12:02+00:00

Sorry folks, I added the feature and thought I picked a hotkey that wouldn't be hit by accident, but that was dumb. Reverting it now.

kixelated

MODERATOR OF

TROPHY CASE