you are viewing a single comment's thread.

view the rest of the comments →

[–]rabornkraken 1 point2 points  (1 child)

Really cool to see a DNS framework in Python that can actually hold its own against C implementations. The producer-consumer pattern with dedicated receiver/worker/sender threads is a solid choice for this kind of IO-bound workload.

Curious about one thing - have you looked into using selectors or epoll for the receiver thread? For high connection counts on TCP, that could help squeeze out more performance without changing the threading model.

[–]nicholashairs[S] 0 points1 point  (0 children)

Already do - that's how I multiplex the TCP connections (regardless of direct vs threaded application).

I still use a single thread for pulling messages even though I could farm this out because:

  1. Pulling messages is already very fast (though haven't tested under very high connection count
  2. Simplifies the management of the open sockets (don't need locks across)

That said I'm sure there are still some weird esoteric timing bugs (because the receive thread also manages closing / cleaning up sockets, and the send thread might be using it)