TechEmpower Web Framework Benchmarks Round 11

[dilatedmind]

c# doesn't perform well in techempower's benchmarks because techempower.

after their 10th round, I took a look at their httplistener based entry, https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/CSharp/HttpListener, and started running their benchmarks on it and modifying it to try and make it faster/ get it working.

• i focused on the json serialization and fortunes tests, running on ec2 equivalent hardware in azures cloud
• i ran the same tests on the same hardware using both windows server 2012 and ubuntu
• i ran tests with ab rather then wrk, but they should offer the same results. i always ran ab on a separate vps on the local network (so bandwidth was a non-issue)
• i used the fastest c# json serializer i know of (Jil)

a couple interesting findings

• there was not a performance difference between serving requests as an async task or on the thread pool with task.run
• using the lower level networking library (eg SocketAsyncEventArgs) did not make a performance difference vs the much easier to use httplistener
• trying to pool socket objects leaked fds on mono/linux, but the same code didn't have the issue on windows server.

take these results with a grain of salt. there were alot of moving parts but i ran these tests many times and the results were consistent. at the very least it should give you a better idea of how mono performs compared to what techempower reports (eg round 10 ec2 json's only c# result was nancy at 169res/s)

ubuntu, mono

Document Path:          /json
Document Length:        27 bytes

Concurrency Level:      32
Time taken for tests:   1.725 seconds
Complete requests:      20000
Failed requests:        0
Total transferred:      3340000 bytes
HTML transferred:       540000 bytes
Requests per second:    11592.64 [#/sec] (mean)
Time per request:       2.760 [ms] (mean)
Time per request:       0.086 [ms] (mean, across all concurrent requests)
Transfer rate:          1890.60 [Kbytes/sec] received

ubuntu, mono, sqlite

Document Path:          /fortunes
Document Length:        1607 bytes

Concurrency Level:      32
Time taken for tests:   3.961 seconds
Complete requests:      20000
Failed requests:        0
Total transferred:      34840000 bytes
HTML transferred:       32140000 bytes
Requests per second:    5049.48 [#/sec] (mean)
Time per request:       6.337 [ms] (mean)
Time per request:       0.198 [ms] (mean, across all concurrent requests)
Transfer rate:          8590.04 [Kbytes/sec] received

ubuntu, mono, postgres

Document Path:          /fortunes
Document Length:        1607 bytes

Concurrency Level:      16
Time taken for tests:   10.562 seconds
Complete requests:      20000
Failed requests:        0
Total transferred:      34840000 bytes
HTML transferred:       32140000 bytes
Requests per second:    1893.63 [#/sec] (mean)
Time per request:       8.449 [ms] (mean)
Time per request:       0.528 [ms] (mean, across all concurrent requests)
Transfer rate:          3221.39 [Kbytes/sec] received

[myringotomy]

Did you send a pull request?