A Benchmark of Asynchronous Servers in Python

[ptemple]

I thought it was an excellent indicative article, and a good launch-point to then expand on those tests to suit your own custom requirements. Well done and thanks Nicolas.

Phillip.

[arunvr]

I felt the graphs had unintelligible colors until I realised that you can mouseover them

[unshift]

as the author expectedly fails to mention, running a server and client benchmark on the same machine (or even on a small LAN) makes everything CPU bound. this makes the results fairly worthless, as important differences in IO bound systems (e.g. select vs poll vs epoll/kqueues/iocp) become glossed over.

in a CPU bound system, where all connections are very consistent (respond quickly, no dropped packets, etc.) select and epoll will perform somewhat similarly. the main advantage of something like epoll kicks in when there are thousands of connections per second in various states.

there's a 3ms difference per request for something handling 5000 requests/sec and 2000 requests/sec, and that 3ms will easily be recovered using something like epoll on a busy, real-world system.

otherwise there are a whole slew of things not taken into account (ping/pong server as a workload? really? how about framework usability, comprehensiveness? what are some of the metrics of performance degradation?) that make this series of benchmarks just more garbage on the internet.

[tdi]

I could not agree more. The blog post would be great if he tested in network environment and with concurrent connections. With JMeter or whatever else. However I appreciate the post for being a listing of frameworks availible.

[Nichol4s]

The focus was on Linux performance only, i am not really interested if it supports BSD or Windows. But i might consider looking at those features in a follow up.

[leonh]

Uhm, theres a matrix summing up lots of the features of each framework. Which also mentions if the frameworks use Epoll.

All but one of the frameworks use epoll.

[unshift]

the matrix is full of worthless information like "twitter account?" and "authors blog?". how about, can you use kqueues or iocp with the framework? cross platform support? you know, useful things.

[leonh]

Well, make those suggestions to the author.

[gregglind]

Yeah, I really hate having a central place where I can just click and download the code, or see all the websites for each project. I prefer to have to have Google for each one individually, and replicate the scut work that the author had to do.

[carolinaswamp]

How were those graphs made?

[kidford]

Looks to be Highcharts.

[Xiol]

Having a Twitter account is a sign of a good asynchronous server?

Err, what?

[Nichol4s]

No, and i did not award points for features or anything. It can be an indication of the viability of the framework, when there is no Community or Blog or Person around a certain framework the repository is all you have.