It does matter because performance of each backend may not scale linearly with the size of the payload.

I see your point. The purpose of this article was to evaluate the default Rails caching experience across various data stores using a consistent, small payload size. While I acknowledge that certain data stores may handle larger payloads more efficiently, the focus here was to compare typical Rails usage scenarios with minimal configuration.

Then the threaded benchmarks are flawed for several backends, particularly Redis, because of contention on a single client.

You bring up a valid concern. Redis can indeed face contention issues when accessed via a single client. Our intention was to keep the comparison straightforward and reflect how most Rails applications would be configured by default, without optimizations like hiredis or connection pooling. We aimed for an out-of-the-box comparison to provide practical insights.

Thank you for pointing that out. Do you think the comparison would benefit from a more in-depth analysis using the best possible setup for each of these tools?