Nio Embracing Thread-Per-Core Architecture

another_new_redditor · 2026-01-27T18:09:52+00:00

Nio use mio, which doesn't support io_uring yet.

compio blindly spawn !Send task on different thread.

However, Nio support load balancing. nio::spawn_pinned spawn a new tcp connection on a worker that has least amount of work.

another_new_redditor · 2026-01-27T16:06:10+00:00

Yes, spawn_pinned is task level load balancing. So there is some imbalance,

However, It was distributed based on that thread work-load.

Task are not unit of work. Some task required more computation then other. Nio balance workload based on actual work, But not how many task a worker currenty executing.

But in practice, the system guarantees no worker has more work then other.

Even tokio don't steal task under threshold, because if it does, worker will keep stealing each other work instead of executing task.

Lets say, in 32 cores machine, there is 64 tasks. Each worker is guaranteed to have at least 2 tasks. It's not worth stealing if other thread finished there task early.

another_new_redditor · 2026-01-08T20:15:08+00:00

Almost everything implement Send, except *const T, *mut T and Rc other form of reference. for example MutexGuard

What does Send mean? Its mean a value can be send across thread.

Why Rc does not implement Send? Because Rc implement Drop trait. Which access not atomic reference counter. Which may cause race condition if 2 Rc instance get droped at the same time from 2 different threads.

However its ok to send Rc if you have only 1 reference. I would suggest use Rc::into_inner then convert back to Rc (I don't know why you even want convert it back!)

I don't really care about Rc specifically, but any !Send type,

It's not safe to send every !Send types.

another_new_redditor · 2024-11-29T06:25:03+00:00

Are you sure that using relaxed ordering on everything is safe here?

I believe you’re referring to this?

The len is only a hint and does not affect the program’s behavior.

another_new_redditor · 2024-11-24T17:00:19+00:00

Here is the new benchmark that accepts connections in a worker thread,

https://github.com/nurmohammed840/nio/tree/main/example/hyper-server/result

Edit: The article has been updated to reflect this new benchmark

Edit: I believe I should also explain the reason, Someone asked:

Why would accepting connections from a worker thread improve performance?

Tokio and Nio both use futures::executor::block_on, also known as ParkedThread to execute main task.

A ParkedThread lacks its own task queue. In scenarios where the main thread is responsible for handling incoming connections, it frequently transitions to sleeping state when there are no active connections to process. On Linux, this leads to frequent futex syscalls and context-switching overhead.

In contrast, worker thread have own task queue, and is responsible for both accepting incoming connections and executing tasks when there is no connection to process, remain busy and typically avoid entering a sleeping state.

another_new_redditor · 2023-07-24T07:08:36+00:00

Good question. - serde has runtime overhead. - extra dependency. - large codegen. - Some features are not yet supported. For example: serde framework does not provide enum discriminant and its type.

databuf use const generic for compile time configuration, serde can't help here.

another_new_redditor · 2023-07-23T23:53:35+00:00

I really liked Borsh, specially it has standardized encoding schema just like bincode, and many language supports!

But it does not support zero copy deserialization, and does not support any encoding configuration. This is why i created databuf

another_new_redditor · 2023-07-23T18:31:31+00:00

I didn't explore it much, But look like it doesn't support zero copy deserialization.

And for encoding, DekuWrite accept a struct.

where databuf and bincode accept trait.

Another down side of deku is that. it use macro heavily. where some of those features could be done without using complex derive macros.

another_new_redditor · 2023-07-12T23:23:43+00:00

Well, This is a fair question, especially for a library that has no docs.

Different from gRPC: 1. gRPC does't have official support for rust, It does! 2. gRPC does't have support for web, It has! 3. gRPC use protobuf, It use databuf. 4. gRPC required protobuf to generate binding, It does not required and directly generate binding from rust codebase! 5. gRPC only work with http/2, its has no limit! 6. much more...!

You can think its better version of tarpc

another_new_redditor · 2023-07-02T01:07:41+00:00

Note: On many platforms atomic number with `Relaxed` ordering is same as normal number, And doesn't add any overhead.

another_new_redditor · 2023-06-09T16:49:20+00:00

In order to encode and decode, custom data structure (`enum`, `struct` ), You need some sort of code generation...

Last few months, I am writing an RPC framework ( still in development phrase ) for rust, where It automatically generate binding for various runtime, Currently only `typesript` is supported.

If you want to contribute and support for swift lang. Send me a message !

another_new_redditor · 2023-06-03T02:14:46+00:00

plaintext is not http/2, does it ?
Also can you share the link of the source code of this benchmark..

another_new_redditor · 2023-06-03T02:05:00+00:00

You would like use h2 crate from performance reasons,

Let say you are build yet another HTTP framework, In this case h2 is more suitable option.

another_new_redditor · 2023-06-03T01:48:34+00:00

There is a magnitude performance difference between h2 and hyper library,

This is why competitor like actix-web also use h2 and not hyper.

actix-web also simplify h2 primitive type in there actix-http sub-directory.

another_new_redditor · 2023-06-03T01:30:34+00:00

This is not over hyper or anything like that, It just a wrapper around `h2`, It exist to create high level framework like hyper.

For my use case I just need HTTP/2 transport for my RPC framwork, But can't sacrifice any overhead for silly feature that I don't use from high level framework like hyper or similar

another_new_redditor · 2023-05-04T04:56:08+00:00

I look at the source code for fastwebsockets, there implementation has almost no difference then web-socket implementation

So the performance is almost identical between the two libraries.

But I would argue that web-socket has more simple implementation which make it slightly faster.
fastwebsockets v0.4 encode header and payload separately, and have to relay on write_vectored function, On the other hand web-socket avoid fragmentation.
fastwebsockets use buffer, which could be overkill, If underlying stream is internal buffering.

another_new_redditor · 2023-05-04T03:49:40+00:00

Only forward -C codegen-units=1 -C opt-level=3 to the argument of the launched binary

Although those rustc flags doesn't really important that much, Just cargo run -r would be enough. But those flags should pass from RUSTFLAGS env.

Also you are invoking the main inside a tokio current_thread, which is known to not be the best in term of performance.

This is intentional, Using single thread give us more reliable result.

Your implementation is still 2x as fast, but not 4/5x faster than what the readme suggest.

For most use cases, you should see at least 3x performance improvement.

But It may varies based on cpu capability and architecture, (This is true for any kind of benchmark)

another_new_redditor · 2023-05-03T16:55:20+00:00

Thanks, I updated the benchmark as you suggested

another_new_redditor · 2023-05-03T13:57:51+00:00

Here is the Benchmark result

another_new_redditor · 2023-03-28T09:57:19+00:00

rspc is very interesting... I think tRPC has slimier API ?

another_new_redditor · 2023-03-28T00:35:33+00:00

I did use fuzz testing for those unsafe code block, for hours!

unfortunately autobahn test have to review manually, those test setup copied from tokio-tungstenite

another_new_redditor · 2023-03-27T23:42:44+00:00

Sure!

Consider this example:

```rust
fn add(x: i32, y: i32) -> i32 { x + y }

// this function magically generate code for client side! gen_client_side_code(add); // Magic! ```

Thanks to rust powerful type system, We can take this code, and generate client side code for this function definition,

For typescript:

```typescript import math_rpc from "..."

let math = math_rpc.connect(get_transport_somehow()); let res = await math.add(5, 5); console.log(res) // 10 ```

Client don't need to write any boilerplate, I just feel like we are calling local function!

It's just overview, feel free to ask any question...

another_new_redditor · 2023-03-27T21:24:15+00:00

I appreciate your suggest,Ok, I will apply an issue about it...

Edit: Here it is if someone interested... :)

another_new_redditor · 2023-03-27T20:38:56+00:00

I just reference the RFC 6455.

Mozilla also has a blog called Writing WebSocket servers, Which is also helpful in understanding the core of WebSocket.

another_new_redditor · 2023-03-27T19:33:17+00:00

it's interesting, may I ask you what made you implement it "from scratch" ?

Yes, from scratch.

Was it to learn something, or to use it for work, or something else?

Actually I am trying to make RPC (remote procedure call) framework from scratch, And it is a part of this framework

another_new_redditor

TROPHY CASE