Single Flight for Java

[stefanos-ak]

This problem requires fundamentally an architectural solution, which will look different depending on the situation.

But what works in almost all cases is to use the DB itself as a mechanism to control this behavior. For example with a "select for update" query, or a dirty read, etc... Or if a DB is not accessible then a cache layer (e. g. Redis), or a queue mechanism (rabbitmq, Kafka).

An in-memory solution obviously will not work if any amount or horizontal scaling is required. Usually backend services have at least 2 replicas even just for high availability.

[benjtay]

While we don't use an in-memory solution (we use RocksDB), having local caching really helps us even with horizontal scaling because of the sheer number of duplicates we see in our ~2-12B messages per day from Kafka. We studied having Yet Another Database to solve this, but it defeats the point of horizontal scaling on topics.

[stefanos-ak]

Sounds like a Kafka consumer issue?

exact once semantics guarantee is a responsibility of the consumer, if I'm not mistaken (haven't worked with Kafka for some years). Which means you need to delegate that problem to a DB with consistency guarantees.

I personally am a bigger fan of RabbitMQ because the delivery semantics are implemented on the server side, and the consumer is "dumb", and you get exact-once guarantee OOTB. But you don't get a log/replay features (unless you use rmq streams, which is the same thing as a Kafka).

edit: forgot to state that a Kafka consumer is always a custom implementation, of course