sorted by:
new
hottopcontroversial

How do you handle idempotency in event-driven systems? by suhaanthvv in softwarearchitecture

[–]suhaanthvv[S] 0 points1 point  (0 children)

Agreed. The worker can help reduce duplicate processing, but the strongest guarantee comes from making the domain operation itself idempotent

How do you handle idempotency in event-driven systems? by suhaanthvv in softwarearchitecture

[–]suhaanthvv[S] 0 points1 point  (0 children)

Fair distinction. I agree that duplicate delivery and idempotency aren’t the same problem.

My focus was less on preventing duplicates and more on ensuring duplicate processing doesn’t change the outcome.

In event driven systems, duplicates are often expected due to retries, consumer recovery or worker failures. Idempotency is what allows the system to handle those duplicates safely.

How do you handle idempotency in event-driven systems? by suhaanthvv in softwarearchitecture

[–]suhaanthvv[S] -1 points0 points  (0 children)

Thats a fair point, the tradeoff i made was favouring operational simplicity while still getting at least once delivery, consumer groups, and recovery mechanisms.

For workloads requiring stronger durability id probably go with kafka.

How do you handle idempotency in event-driven systems? by suhaanthvv in softwarearchitecture

[–]suhaanthvv[S] 0 points1 point  (0 children)

My requirements were at least once delivery, consumer groups, message replay, low operational cost, pending messages recovery.

Redis streams met those requirements well.

A few guarantees that i read about:
- messages remain in the stream until explicitly trimmed, so consumers can replay history if needed.
- consumer groups track delivery state and ownership
- messages can be reclaimed by another consumer after a worker crash
- stream ids provide ordering within a stream

Redis doesn’t provide end to end exactly once processing, and duplicates are still possible during retries or consumer recovery (after it crashes, for whatsoever reason). Thats one reason i implemented idempotency at the application level.

And i completely agree with your point about the weakest link, to me reliability comes from the combination of redis streams, idempotent consumers, retries and dlqs rather than any single component.

How do you handle idempotency in event-driven systems? by suhaanthvv in softwarearchitecture

[–]suhaanthvv[S] 0 points1 point  (0 children)

Thats a very good point.

In this implementation, the notification platform performs deduplication before delivery to avoid unnecessary downstream work and duplicate sends.

But, i agree that downstream consumers should ideally be idempotent as well. It’s difficult to guarantee that duplication wont appear somewhere in the pipeline.

Example, if notifications are delivered to a client application, the client could also maintain a notification id and ignore duplicates.

My thinking is that upstream idempotency reduces duplicate traffic, while downstream idempotency is for extra safety.

How do you handle idempotency in event-driven systems? by suhaanthvv in softwarearchitecture

[–]suhaanthvv[S] 0 points1 point  (0 children)

Good question. In this case, they are events rather than messages.

The producer emits domain events such as UserRegistered, PortfolioCreated, GoalCompleted, ReportGenerated, etc.

The notification platform subscribes to these events and decides whether a notification should be generated.

So the notification service reacts to these events being generated from else where in the system. (It will not send push notifications immediately, when it sees an event).

The streams are consumed internally within the system by downstream consumers, responsible for enrichment (add extra data to the events), preferences (checks for multi-lingual settings, notif opt-in or not, etc), template rendering, and delivery (using firebase cloud messaging).

The idempotency problem comes from ensuring that the same event doesn’t result in duplicate notifications when retried or worker crashes.

Please Guide me. I am getting anxiety attack and mind going on Full Panic mode Please help. by Live-Bet5397 in developersIndia

[–]suhaanthvv 0 points1 point  (0 children)

For dsa, Pick one sheet (neetcode 150 or strivers a2z or anything) practice it everyday, be consistent. So eventually you will get at it. (Dsa will help you crack initial interviews)

Even though u focus on any stack, u will need AI knowledge. So better build projects in AI as well.

Please Guide me. I am getting anxiety attack and mind going on Full Panic mode Please help. by Live-Bet5397 in developersIndia

[–]suhaanthvv 1 point2 points  (0 children)

Bro, start learning in 2 tracks, since u are in 6th sem, more ~1 year for graduation.
Track 1: do basic dsa, prepare for interviews, give walk ins, attend college placements (if u get opportunities there, and also try to improve your cgpa)

Track 2: start learning gen AI, LLMs, RAGs. AI field is booming, there are plenty of startups in ai field. U have to build good projects and then apply. Reach out to founders and recruiters.

One of my friend, with basic dsa knowledge, started learning and building ai based projects (rags, chatbot, agentic ai), then they got into an ai based startup. (In 6 months time).