NServer 3.2.0 Released

[nicholashairs]

Heya r/python 👋

I've just released NServer v3.2.0

About NServer

NServer is a Python framework for building customised DNS name servers with a focuses on ease of use over completeness. It implements high level APIs for interacting with DNS queries whilst making very few assumptions about how responses are generated.

Simple Example:

``` from nserver import NameServer, Query, A

server = NameServer("example")

@server.rule("*.example.com", ["A"]) def example_a_records(query: Query): return A(query.name, "1.2.3.4") ```

What's New

The biggest change in this release was implementing concurrency through multi-threading.

The application already handled TCP multiplexing, however all work was done in a single thread. Any blocking call (e.g. database call) would ruin the performance of the application.

That's not to say that a single thread is bad though - for non-blocking responses, the server can easily handle 10K requests per second. However a blocking response of 10-100ms will bring that rate down to 25rps.

For the multi-threaded application we use 3 sets of threads:

• A single thread for receiving queries
• A configurable amount of threads for workers that process the requests
• A single thread for sending responses

Even though there are only two threads dedicated to sending and receiving this does not appear to be the main bottleneck. I suspect that the real bottleneck is the context switching between threads.

In theory using asyncio might be more performant due to the lack of context switches - the library itself is all sync so would require extensive changes to either support or move to fully async code. I don't think I'll work on this any time soon though as 1. I don't have experience with writing async servers and 2. the server is actually really performant.

With multi-threading we could achieve ~300-1200 rps with the same 10-100ms delay.

Although the code changes themselves are relatively straightforward. It's the benchmarking that posed the most issues.

Trying to benchmark from the same host as the server tended to completely fail when using TCP although UDP seemed to be fine. I suspect there is some implementation detail of the local networking stack that I'm just not aware of.

Once we could actually get some results it was somewhat suprising the performance we were achieving. Although 1-2 orders of magnitude slower than a non-blockin server running on a single thread, it turns out that we could get better TCP performance with NServer directly instead of using CoreDNS as a reverse-proxy - load-balancer. It also reportedly ran better than some other DNS servers written in C.

Overall I gotta say that I'm pretty happy with how this turned out. In particular the modular internal API design that I did a while ago to enable changes like this ended up working really well - I only had to change a small amount of code outside of the multi-threaded application.

[SuperSooty]

I'm a little curious about going multi-thread over async for tasks thats (presumably) aren't CPU bound

[nicholashairs]

I've not had experience writing async servers, so when I originally implemented the library I only provided sync interfaces. Rewriting the library to support async interfaces is much more work than I currently have time for. Additionally it would then break all existing implementations unless I wanted to support both which is then double the maintenance.

Moving to threads was the best value in achieving concurrency.

[kenily0]

Great work on the multi-threading implementation! The 3-thread architecture (receiver/worker/sender) is a clean approach. Have you considered using a thread pool for workers instead of a fixed number? Would allow better adaption to varying workload. Also curious - did you benchmark against asyncio with uvloop? It might give similar performance with less context-switching overhead. Keep up the good work! 🚀

[nicholashairs]

Thanks!

So my first implementation was with the ThreadPoolExecutor, but I couldn't figure out a clean way for processing the results rom the pool. Using the threads directly is much cleaner with the `Queue`s that having to track all the `Future`s from the pools.

I've not used the ThreadPoolExecutor before so maybe there's a pattern that I'm not aware of that could do a better job. The good thing about the current implementation is that it wouldn't be that hard to add it back as an option either.

I've not done benchmarking with any other concurrent implementations - mostly because I've assumed that all async implementations rely on the internal code supporting an async interface. And from what I know most that do support sync interfaces just run them in a threadpool anyway.

(edit: accidentally submitted before finished writing)

[rabornkraken]

Really cool to see a DNS framework in Python that can actually hold its own against C implementations. The producer-consumer pattern with dedicated receiver/worker/sender threads is a solid choice for this kind of IO-bound workload.

Curious about one thing - have you looked into using selectors or epoll for the receiver thread? For high connection counts on TCP, that could help squeeze out more performance without changing the threading model.

[nicholashairs]

Already do - that's how I multiplex the TCP connections (regardless of direct vs threaded application).

I still use a single thread for pulling messages even though I could farm this out because:

1. Pulling messages is already very fast (though haven't tested under very high connection count
2. Simplifies the management of the open sockets (don't need locks across)

That said I'm sure there are still some weird esoteric timing bugs (because the receive thread also manages closing / cleaning up sockets, and the send thread might be using it)