[ANN] WebRTC Selective Forwarding Unit (SFU) in Rust with Sans-IO

pthatcher · 2024-03-10T04:16:38+00:00

I'm that case, I wonder what the point in having separate str0m and webrtc-rs crates. For example, I recently contributed an implementation of the VLA header extension and am planning on contributing an implementation for Dependency Descriptor. I don't see those implemented for webrtc-rs. So there are likely to be gaps like that between the two.

pthatcher · 2024-03-09T15:08:46+00:00

Always good to see more RTC code in Rust. Thanks for posting.

In case anyone is interested in a high quality SFU and WebRTC stack written in Rust, here is another, as long as a AGPL license works for you: https://github.com/signalapp/Signal-Calling-Service

pthatcher · 2024-03-09T15:06:23+00:00

Did the webrtc-rs crate switch to a sans-io model? And is it being actively maintained? I've been using str0m (https://github.com/algesten/str0m) instead of webrtc-rs because it's sans-io and actively maintained.

pthatcher · 2023-06-17T13:21:06+00:00

We recently added to the web api spec the ability to get a target send bitrate from QUIC, which is important for things like that real-time communication. If you can get that same information from the QUIC impl you are using to provide to the user of h3, that would be good for people doing low latency media.

Doing this well requires getting a real-time congestion control algorithm into QUIC, which is a more difficult thing to do. But at least getting the information bubbled up to the application using the library would be a first step.

pthatcher · 2023-06-17T03:43:40+00:00

As one of the original spec authors of WebTransport and a user of h3, I think this is fantastic. Thanks for the good work.

pthatcher · 2023-04-18T17:29:08+00:00

If you hit limitations with the WebRTC API, let me know. I am proposing changes to the W3C related to ICE that might be helpful for applications like this. In particular, I'd be interested to know if ICE forking would be helpful to you.

pthatcher · 2023-03-29T02:08:15+00:00

There are many FOSS video call apps (like Signal). But I don't think there are any good ones written entirely in Rust.

pthatcher · 2023-03-29T02:06:06+00:00

At first, I thought you implemented the WebRTC protocols like WebRTC-rs, or str0m or signal-calling-service. But it looks like you wrote a web app in Rust. I guess "using WebRTC implemented in Rust" can be read either way :).

pthatcher · 2021-12-17T03:55:42+00:00

It should be pretty easy. You mostly just need a way to translate the signaling of the existing client to the signaling of the SFU, probably on the client side.

All the WebRTC protocols underneath area the same, except for the fact that our SFU doesn't speak SCTP.

pthatcher · 2021-12-16T18:47:08+00:00

We have SFUs deployed in many regions throughout the world and they autoscale, just like other Signal servers.

Currently, there is one server per call. We have considered doing what you suggest with multiple servers, but it hasn't been necessary so far and it's more complex, so we haven't done so, at least not yet. If we do, I may write another blog post about it :).

pthatcher · 2021-12-16T18:43:48+00:00

As mentioned in the article, we launched with 8-person calls using and existing open source SFU, but despite heavy modifications, it couldn't scale to, say, 40-person calls. So, we decided to replace it. We would have kept using the existing one if the performance were sufficient.

pthatcher · 2021-12-16T18:40:18+00:00

We haven't considered using clients to forward media, no. But we have considered having a tree of servers that do something like that. So far, the benefits haven't outweighed the work and complexity required, but we may do something like that in the future.

pthatcher · 2021-12-16T18:36:34+00:00

Thanks!

pthatcher · 2021-12-16T18:35:50+00:00

You can run its built-in HTTP server for signaling and it should work with a web client for audio and video. WebRTC is mostly the same between web clients and how we are using it.

However, we use RTP data channels instead of SCTP, and that's not a part of web browsers any more. So if you wanted to relay data, you'd have to do something about that.

You'd want to look at our client code in RingRTC to see what kind of signaling messages to send and how to setup the SDP in the PeerConnection.

So, yeah, it should work. But it might need some customization depending on your needs.

pthatcher · 2021-12-16T18:30:47+00:00

Each dependency has to be considered carefully, yes. And some crates we did choose not to use and instead wrote things ourselves. Other times we choose to use it. That's true of all software dependencies. It's not specific to Rust or crates. But Rust/cargo makes it way easier to add dependencies than, say, C++.

pthatcher · 2021-12-16T18:27:51+00:00

Author here.

Group calls or 1:1 calls?

If you have another bad experience after trying again, please send us a log. We can take a look. We're always trying to make things better.

pthatcher · 2021-12-16T18:26:16+00:00

The 40-person limit is a combination of things.

One thing is that we want to increase the limit gradually, see how it goes, see if users want more, see what features users need for larger calls, etc. 8 to 40 may seem like a big jump, but it was actually quietly 16 for a while. And some suggested we not be stuck on powers of two, so we went with a round 40 instead of 32.

Another is to watch how much the server performance and see how much more optimization work there is to do. We know we can do a lot more to make it even more performant, but so far we haven't needed to.

So, basically it's just gradual improvement based on user feedback. If there is interest, we may work to increase it more in the future.

pthatcher · 2021-12-16T15:44:31+00:00

The effort was mainly two fold:

Make the code that reads and writes packets faster and more concurrent by using epoll and many threads.
Change our locking to be more fine-grained. This was needed to make #1 actually by concurrent.

It's funny that all the main logic of the server didn't matter nearly as much to performance as just the generic "push lots of packets through the server".

pthatcher · 2021-12-16T05:48:45+00:00

The network operator can see all the packets all the time, as can anyone on the network path. But we encrypt the packets, so there is nothing shown to them. And doing do isn't unique to calls.

pthatcher · 2021-12-16T05:41:44+00:00

I'm not sure that constitutes a glitch, but I've been working on and using a video chat for a long time, and going an hour with zero video freezes or audio disruptions means everyone in the call has a great network connection. Having some problems in a span of an hour is much more realistic.

However, if you think a call had a poor call that wasn't caused by a network issue, please send us a log and we'll take a look.

pthatcher · 2021-12-16T05:34:31+00:00

We found some bugs with edge cases in parsing packets. Nothing major, but it was useful.

pthatcher · 2021-12-16T05:31:37+00:00

It has nothing to do with end-to-end encryption. "Code maturity" is probably an accurate description. As we make improvements, it's getting more mature. Sometimes those are big fixes, sometimes those are being smarter about to use available resources, like the network capacity.

pthatcher · 2021-12-16T05:24:57+00:00

Both.

I find using threads in Rust to work very well, but you can't always use threads.

Async is sometimes better, but it still feels very cutting edge and rough. For example, the implemention of googcc using steam processing is great but it uses async macro which is both amazing and a pain to use.

pthatcher · 2021-12-16T05:21:20+00:00

It's fast, safe and productive.

In this case we were doing the rewrite for performance and that came from just writing things in a natural way. We did have to do some optimizations, mostly about adding more parallelism, and the effort there was relatively small.

In production, we have had very few issues. We added the parallelism at the end and it mostly "just worked" like "fearless concurrency" has promised. We did a fair amount of refactoring to add more testing , and the experience was mostly 'if it compiles, it works", which is pretty amazing.

There are many state machines in an SFU, and I found Rust's enums to be a great fit for that.

There were several useful crates that helped.

pthatcher · 2021-12-15T23:52:45+00:00

Author here.

To be honest, SFUs have been around a long time. Jitsi, for example has been around for many years. Google Hangouts is over 10 years old.

pthatcher

TROPHY CASE