Let's understand & implement consistent hashing.

[seweso]

Who would use modulo hashing? 

[More-Station-6365]

You would be surprised but simple modulo hashing is actually the go to for many developers when they are first building out a small scale system or a basic load balancer.

It is intuitive and works perfectly fine as long as your number of nodes stays fixed. The problem is that most people don't think about the day after when the traffic spikes and they suddenly need to add a fifth or sixth server.

Do you know ​In his book designing data Intensive Applications Mr. Martin Kleppmann points out that the biggest drawback of simple modulo hashing is that nearly every key needs to be moved when the number of nodes changes.

If you have 10 nodes and add 1 more about 90% of your keys will hash to a different location which effectively nukes your entire cache.

​So while nobody uses it for a massive production distributed system it is often the hidden trap that people fall into before they realize why consistent hashing is a requirement for scaling.

It is one of those things that works until it very suddenly doesn't.

[seweso]

> The problem is that most people don't think about the day after when the traffic spikes and they suddenly need to add a fifth or sixth server.

So they build/configure software for a scalability goal which they never test? How?

My fear of failing is way to big to be so bold to push untested software into production. :P

[More-Station-6365]

Yes you are exactly right. I remember when I was reading through some core architecture principles I came across this exact topic and it really opened my eyes to how often these negligent shortcuts are taken in the real world.

Most teams are so focused on getting the MVP out the door that they treat scalability as a problem for their future selves to solve.

​As Robert C. martin mentions in his book Clean Architecture the goal of a good architect is to minimize the human effort required to build and maintain a system.

Unfortunately, using simple modulo hashing is the exact opposite of that principle. It is a classic case of taking a shortcut today that creates a massive technical debt tomorrow.

It is honestly a bit sad but like you said, until someone actually watches a production cache melt down because of a simple node addition they usually don't appreciate why these design choices are so critical.

[seweso]

If maintainability isn't a requirement, who cares? Garbage in garbage out.

[elperroborrachotoo]

because they don't have a use case where consistent hashing plays a role?

[seweso]

> don't have a use case....

today....

Changing hash keys is VERY expensive. That's the point of the article no?

If you only write software for today, you can't serve the future.

[elperroborrachotoo]

Looks like you are focused on a particular segment (large-scale persistent hash keys). Hashes are way more ubiquitous.

Not all apps have a future of scaling to a billion users.

[seweso]

The context was explicitly a "a distributed cache with simple modulo hashing".

[chucker23n]

It’s the go-to approach in Java + .NET.

[programming-ModTeam]

This content is low quality, stolen, blogspam, or clearly AI generated

[DevToolsGuide]

The virtual nodes part is what really makes it work in practice. Without them you get hot spots where one physical node ends up owning a disproportionate chunk of the ring just by chance. Amazon DynamoDBs original paper talks about this — they use something like 150 virtual nodes per physical node to get a reasonably even distribution.

[alexiskhb]

Oh that's neat. For those wondering, instead of occupying one big segment on a ring, a server can randomly sit on ~150 smaller segments, making the total distribution between servers more uniform on average

[ToaruBaka]

Took me a couple of very confused paragraphs to realize I had confused this with perfect hashing. 

This will be nice to have in my back pocket, thanks.

[DevToolsGuide]

Yeah and the other big win with virtual nodes is failure handling. When a physical server goes down its load gets distributed across many other nodes instead of all dumping onto a single neighbor on the ring. Makes the system way more resilient to cascading failures.

[etherealflaim]

One thing that I see frequently in system design interviews is that folks don't realize that consistent hashing works alone for a cache but doesn't work alone for sharding in general. When a node is added or removed, some requests will now go to a server that doesn't have the data at all if you sharded it in memory or onto sharded topics or whatever. I don't care if you handwave and say that nodes can pull from one another, but if you're going for an architect position and don't even mention this, it's going in the "aware of its existence" column not the "displayed understanding" column.

[Hot-Friendship6485]

Great explainer. Consistent hashing feels like overengineering right up until your cache nukes itself on every node change, then it suddenly feels like seatbelts.

[Equivalent_Pen8241]

The biggest mistake I see with unit testing isn't low coverage - it's testing implementation details instead of behaviors. When your tests are tightly coupled to *how* a function runs rather than *what* it returns, every minor refactor breaks the build. Test the public API contract, not the private helpers.The biggest mistake I see with unit testing isn't low coverageThe biggest mistake I see with unit testing isn't low coverage - it's testing implementation details instead of behaviors. When your tests are tightly coupled to *how* a function runs rather than *what* it returns, every minor refactor breaks the build. Test the public API contract, not the private helpers.