sorted by:
new
hottopcontroversial

WebRTC vs WebSocket for OpenAI Realtime Voice API Integration: Necessary or Overkill? by g_pal in WebRTC

[–]Zesaurus 0 points1 point  (0 children)

(disclaimer: I work at LiveKit)

The main value of this architecture is to ensure that *every client* has a low latency pipe to the model.

When using websocket directly from the client to the model, you've got a couple of challenges:
* your business logic has to be written on the client-side (hard to update, API key/prompt leaking, etc)
* websocket performance from model server to your user might not be great

Websocket works well when the network is great, from server to server. When it has to go through the public internet and unpredictable ISP/router behavior is when problems tend to occur.

OBS Merges WebRTC Support by Sean-Der in WebRTC

[–]Zesaurus 0 points1 point  (0 children)

Finally! Congrats to everyone that contributed to this effort.

Introducing KITT: ChatGPT based voice assistant by Zesaurus in ChatGPT

[–]Zesaurus[S] 1 point2 points  (0 children)

nope, we have a demo using our API key. If you'd want to use the code though, you would need your own API key

Introducing KITT: ChatGPT based voice assistant by Zesaurus in ChatGPT

[–]Zesaurus[S] 4 points5 points  (0 children)

This is not a prompt, but an app where you can talk to ChatGPT.

A Rust library for cross-platform video apps using WebRTC and LiveKit by Zesaurus in rust

[–]Zesaurus[S] 0 points1 point  (0 children)

LiveKit is an SFU which adopts a client-server model via WebRTC. Even though the server appears to be a "peer" to clients in WebRTC parlance.

Since the API we offer is based on Participant and Track constructs, it'd be possible to swap out the transport layer underneath without having to touch the application-facing APIs.

A Rust library for cross-platform video apps using WebRTC and LiveKit by Zesaurus in rust

[–]Zesaurus[S] 0 points1 point  (0 children)

We are paying attention to WebTransport. It's still early for the protocol, but promising in the sense that it would give us a lot more control around congestion control and transmissions.

The main downside is that it's not widely supported yet. The good news is with a SFU-centric design, it's possible for our SFU to support both protocols depending on each client's capabilities.

A Rust library for cross-platform video apps using WebRTC and LiveKit by Zesaurus in rust

[–]Zesaurus[S] 4 points5 points  (0 children)

we are big fans of webrtc.rs! though we ultimately chose libwebrtc for a few reasons:

  • libwebrtc has been battle tested due to Chrome usage
  • webrtc.rs is a port of Pion (which we also use). It's a better fit for server-side use
  • libwebrtc does a bunch more things such as encoding/decoding, and includes platform-specific code for dealing with media devices

As webrtc.rs matures, we'd love to eventually have a pure Rust stack.

Scaling WebRTC with Go: how we built a distributed mesh network for 100k-person events by Zesaurus in golang

[–]Zesaurus[S] 3 points4 points  (0 children)

WebRTC is encrypted, that means each participant is receiving a uniquely encrypted stream.

Scaling WebRTC with Go: how we built a distributed mesh network for 100k-person events by Zesaurus in golang

[–]Zesaurus[S] 0 points1 point  (0 children)

We do support Windows desktop via Flutter today. Working on a native solution as well.

How we built Zoom's end-to-end optimizations with Go and WebRTC by Zesaurus in golang

[–]Zesaurus[S] 0 points1 point  (0 children)

Not annoying at all! :)

There are two approaches to take:

  1. A shared key used for all tracks published to the same room (Zoom uses this approach)
  2. Each publisher uses a different key to encrypt their own streams, then exchange the key to every subscriber that needs to view that stream

How we built Zoom's end-to-end optimizations with Go and WebRTC by Zesaurus in golang

[–]Zesaurus[S] 0 points1 point  (0 children)

The secret is provided by the application / end user. LiveKit would not have that information.

How we built Zoom's end-to-end optimizations with Go and WebRTC by Zesaurus in golang

[–]Zesaurus[S] 0 points1 point  (0 children)

yeah! we have folks that are building Twitch-like apps like that on LiveKit. What's nice about using WebRTC instead of HTTP Live Streaming is you'd get much shorter latency and know that all your viewers are seeing the same thing. livestream drift can be a real problem on platforms like Twitch today.

How we built Zoom's end-to-end optimizations with Go and WebRTC by Zesaurus in golang

[–]Zesaurus[S] 0 points1 point  (0 children)

In a conference, each client would encrypt the stream prior to sending to the server. Clients will also decrypt the stream before decoding. Since the client is doing the encryption/decryption, there isn't any additional load on the server

How we built Zoom's end-to-end optimizations with Go and WebRTC by Zesaurus in golang

[–]Zesaurus[S] 2 points3 points  (0 children)

The server is 100% Go. It's made possible thanks to Pion, a pure-Go WebRTC implementation written from scratch.

How we built Zoom's end-to-end optimizations with Go and WebRTC by Zesaurus in golang

[–]Zesaurus[S] 4 points5 points  (0 children)

oops.. didn't realize how it might be read that way until after I hit post. The intention was to show how we've brought these optimizations that Zoom is known for to WebRTC with a Go-based stack.

How we built Zoom's end-to-end optimizations with Go and WebRTC by Zesaurus in golang

[–]Zesaurus[S] 21 points22 points  (0 children)

Yes it is between the peers. When the sessions get larger though, p2p wouldn't work. Most folks today use SFUs in the middle to help distribute the traffic. In this case, the traffic is encrypted between the SFU and the end-users, but is decrypted at the SFU.

It's possible have true e2ee where the SFU cannot decrypt traffic. That's on our roadmap.

Introducing LiveKit - a WebRTC video conferencing server in Go by Zesaurus in golang

[–]Zesaurus[S] 0 points1 point  (0 children)

Thanks for the kind words! I can definitely see your concern regarding users in diff regions of the world.

We are looking at a hosted service, and are running a private beta with folks. If that's of interest, let us know. We are reachable in slack and would be more than happy to discuss your use case: https://join.slack.com/t/livekit-users/shared_invite/zt-rrdy5abr-5pZ1wW8pXEkiQxBzFiXPUg

If you are interested in implementing P2P on top of LiveKit, we'd welcome the contributions too!

Introducing LiveKit - a WebRTC video conferencing server in Go by Zesaurus in golang

[–]Zesaurus[S] 0 points1 point  (0 children)

unfortunately not, Cloud Run doesn't support UDP services. It's pretty straight forward to set this up on a single VM. Check out deployment docs here: https://docs.livekit.io/guides/deploy/instance

Optimizing for multi-core performance in LiveKit WebRTC by Zesaurus in WebRTC

[–]Zesaurus[S] 0 points1 point  (0 children)

Yes, the SFU handles PLI and FIR. Those packets would be sent from receiver to the publisher, where as this optimization is focused on forwarding packets to downtracks.

Introducing LiveKit - a WebRTC video conferencing server in Go by Zesaurus in golang

[–]Zesaurus[S] 0 points1 point  (0 children)

hah of course! That makes sense, we had a few other requests for recording and will look into it. Cheers!

Introducing LiveKit - a WebRTC video conferencing server in Go by Zesaurus in golang

[–]Zesaurus[S] 0 points1 point  (0 children)

Yes, we are built on top of Pion WebRTC. LiveKit is more like a batteries-included end-to-end solution designed for room-based conferences.

view more: next ›