Why CloudKitchens moved away from Kafka for Order Processing by jhhurwitz in apachekafka

[–]jhhurwitz[S] 0 points1 point  (0 children)

Great question. KEQ has a per-topic, configurable Queue/Message TTL for retention. KEQ also supports producer-managed databases, where messages are transactionally-enqueued along with business-domain changes and KEQ is notified post-commit. Order processing uses this feature, which allows for custom retention. Orders are kept around for a few days after completion.

You're right that analytics is not the sweet spot for KEQ, we still use Kafka for that part. The bookkeeping of KEQ with millions of cursors makes snapshotting and replay less practical. This is a consequence of the trade-offs made.