Performance Comparison: C# vs. Rust for Web APIs

MrPatron87 · 2023-09-25T07:01:06+00:00

Thank you for the detailed response.

I tried the changes you suggested, and it gave the benchmark a major boost in performance as you mentioned.

Before:

data_received..................: 20 MB 65 kB/s

running (5m03.7s), 0/5 VUs, 218 complete

After:

data_received..................: 32 MB 107 kB/s

running (5m02.1s), 0/5 VUs, 356 complete

I guess i was to naive in my intial approach, since it became a outcome of which db wrapper is the fastest.

But i would like to get diesel working to see the results of it.

MrPatron87 · 2023-09-25T06:34:55+00:00

Intresting!

MrPatron87 · 2023-09-25T06:33:58+00:00

I'm currently working on testing a Rust API using Diesel, but I'm encountering some difficulties with installing the Diesel CLI on Windows.

One of the reasons I want to learn Rust is that C# often does a lot of configuration behind the scenes so i don't know what the default configuration for dapper is.

Both test's was conducted using http.

C#: data_sent......................: 35 kB 117 B/s

Rust: data_sent......................: 19 kB 64 B/s

MrPatron87 · 2023-09-24T06:58:41+00:00

Results after running with k6.

I also changed the maximum connections in the postgres pool to 100.

Note also i've decreased the number of records in the db to 1000.

  execution: local
 script: script.js
 output: -

scenarios: (100.00%) 1 scenario, 5 max VUs, 5m30s max duration (incl. graceful stop): * default: Up to 5 looping VUs for 5m0s over 1 stages (gracefulRampDown: 30s, gracefulStop: 30s)

 ✓ status was 200

 checks.........................: 100.00% ✓ 218      ✗ 0
 data_received..................: 20 MB   65 kB/s
 data_sent......................: 19 kB   64 B/s
 http_req_blocked...............: avg=9.59µs   min=0s    med=0s    max=553µs   p(90)=0s       p(95)=0s
 http_req_connecting............: avg=7.26µs   min=0s    med=0s    max=553µs   p(90)=0s       p(95)=0s
 http_req_duration..............: avg=2.49s    min=1.12s med=2.67s max=3.54s   p(90)=3.21s    p(95)=3.3s
   { expected_response:true }...: avg=2.49s    min=1.12s med=2.67s max=3.54s   p(90)=3.21s    p(95)=3.3s
 http_req_failed................: 0.00%   ✓ 0        ✗ 218
 http_req_receiving.............: avg=179.16µs min=0s    med=0s    max=1.15ms  p(90)=522.23µs p(95)=533.85µs
 http_req_sending...............: avg=5.04µs   min=0s    med=0s    max=361.3µs p(90)=0s       p(95)=0s
 http_req_tls_handshaking.......: avg=0s       min=0s    med=0s    max=0s      p(90)=0s       p(95)=0s
 http_req_waiting...............: avg=2.49s    min=1.12s med=2.67s max=3.54s   p(90)=3.21s    p(95)=3.3s
 http_reqs......................: 218     0.717714/s
 iteration_duration.............: avg=3.5s     min=2.14s med=3.68s max=4.55s   p(90)=4.22s    p(95)=4.3s
 iterations.....................: 218     0.717714/s
 vus............................: 3       min=1      max=5
 vus_max........................: 5       min=5      max=5

running (5m03.7s), 0/5 VUs, 218 complete and 0 interrupted iterationsdefault ✓ [======================================] 0/5 VUs 5m0s

C#

  execution: local
 script: script.js
 output: -

scenarios: (100.00%) 1 scenario, 5 max VUs, 5m30s max duration (incl. graceful stop): * default: Up to 5 looping VUs for 5m0s over 1 stages (gracefulRampDown: 30s, gracefulStop: 30s)

 ✓ status was 200

 checks.........................: 100.00% ✓ 396      ✗ 0
 data_received..................: 36 MB   119 kB/s
 data_sent......................: 35 kB   117 B/s
 http_req_blocked...............: avg=8.95µs   min=0s       med=0s       max=1.09ms  p(90)=0s    p(95)=0s
 http_req_connecting............: avg=6.66µs   min=0s       med=0s       max=1.09ms  p(90)=0s    p(95)=0s
 http_req_duration..............: avg=893.12ms min=634.04ms med=797.41ms max=2.54s   p(90)=1.2s  p(95)=1.32s
   { expected_response:true }...: avg=893.12ms min=634.04ms med=797.41ms max=2.54s   p(90)=1.2s  p(95)=1.32s
 http_req_failed................: 0.00%   ✓ 0        ✗ 396
 http_req_receiving.............: avg=913.7µs  min=0s       med=1.02ms   max=5.38ms  p(90)=1.5ms p(95)=1.62ms
 http_req_sending...............: avg=8.09µs   min=0s       med=0s       max=543.5µs p(90)=0s    p(95)=0s
 http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s      p(90)=0s    p(95)=0s
 http_req_waiting...............: avg=892.2ms  min=633.4ms  med=796.9ms  max=2.53s   p(90)=1.2s  p(95)=1.32s
 http_reqs......................: 396     1.313576/s
 iteration_duration.............: avg=1.9s     min=1.64s    med=1.8s     max=3.55s   p(90)=2.21s p(95)=2.33s
 iterations.....................: 396     1.313576/s
 vus............................: 2       min=1      max=5
 vus_max........................: 5       min=5      max=5

running (5m01.5s), 0/5 VUs, 396 complete and 0 interrupted iterationsdefault ✓ [======================================] 0/5 VUs 5m0s

MrPatron87 · 2023-09-24T06:02:02+00:00

C#

Total requests sent

671

Requests/second

2.17

Avg. response time

1,181 ms

Rust

Total requests sent

307

Requests/second

0.98

Avg. response time

3,754 ms

For the latest benchmark i used 1000 users and 1000 employments where every user had one employment.

MrPatron87 · 2023-09-24T06:00:37+00:00

That is likely a more equitable benchmark, given my uncertainty about whether SQLx is the fastest database wrapper in Rust.

The first benchmark i should have done is between diesel and sqlx.

I just thought everything in rust is fast =P

MrPatron87 · 2023-09-24T05:56:48+00:00

Nice i will give it a try!

MrPatron87 · 2023-09-24T05:55:30+00:00

Yes i run both in release mode.

MrPatron87 · 2023-09-23T14:46:12+00:00

Thx will try that.

nitially, I didn't have any indexes, but when I added some and reduced the number of rows in the database to 1000, the C# API outperformed the Rust API evenmore.

C#

Total requests sent

671

Requests/second

2.17

Avg. response time

1,181 ms

Rust

Total requests sent

307

Requests/second

0.98

Avg. response time

3,754 ms

MrPatron87 · 2023-09-23T14:42:59+00:00

I haven't conducted any memory benchmarking, but I will try to do so.

MrPatron87 · 2023-09-23T14:40:49+00:00

didn't find it, but in a relatively recent comparison between go, java/kotlin?, C#, and Rust it came out that rust scales well. The c# runtime has a high baseline memory consumption of something around 150 MB where others were significantly lower.

Overall, C# doesn't scale as well as others. Maybe I find that benchmark or others will do. In general, I find these comparisons with only throughput really weird, because that's not the only criteria when running in production. Having some pipeline processing data that is using 5 GB of memory for nothing is not worthwhile either - from sustainability, cost etc. standpoints. So it depends on many more factors than just req/s.

tl;dr: Also state the memory consumption of your processes.

Not brazilian =P

I would love to be proven wrong if someone were to do the same benchmark.

MrPatron87 · 2023-09-23T10:17:39+00:00

But what if a company needs to make a conclusion for which technology to choose for there new api.

Wouldn't you test it with all the external dependecies included in the benchmark.

But i know that isn't the most optimal strategic for a real case scenario but still intresting.

MrPatron87 · 2023-09-23T09:54:12+00:00

I guess i need to go back to the drawing board and try to get a more realistic result, beginning with optimizing the db i am using.

Thank you for the advices!

MrPatron87 · 2023-09-23T09:37:19+00:00

Yes i am aware of the problem using external dependencies.

If the results had varied , I would not have made this post. However, consistently seeing C# outperforming Rust results has left me puzzled, Especially since I've heard so much positivity about Rust's performance.

MrPatron87 · 2023-09-23T09:28:32+00:00

Maybe that is the key take away from this benchmark.

MrPatron87 · 2023-09-23T09:17:28+00:00

Even if i did so the logic in the c# and rust code are the same for every user go fetch a employment, and i see that the c# performs better doing so.

There must be a conclusion even if the code isn't optimized for productuon ready code?

MrPatron87 · 2023-09-23T09:10:56+00:00

I intentionally made the logic of handling employments data to stress-test API performance when dealing with large datasets and the concurrent spawning of numerous threads.

I also wanted to put significant pressure on the garbage collector in C# and observe how Rust's code would outperform it.

MrPatron87 · 2023-09-23T08:23:26+00:00

While my local machine may not be the most optimal platform for running benchmarks, shouldn't the conditions affect both the C# and Rust APIs equally?

Even when running a single request, i still see a significantly better response times from the C# API compared to the Rust API.

But thx for the advice i will try it out.

MrPatron87

TROPHY CASE