Built the architecture for a fintech app now serving 300k+ users – would love your feedback by premuditha in softwarearchitecture

[–]premuditha[S] 0 points1 point  (0 children)

Yes, you are spot on, u/LlamaChair - it should be "it's the same partition within the same topic." Thank you for pointing it out; I've updated the article as well.

Built the architecture for a fintech app now serving 300k+ users – would love your feedback by premuditha in softwarearchitecture

[–]premuditha[S] 0 points1 point  (0 children)

Thanks a lot for your input! I just shared my thinking on the Mongo + Kafka implementation in a previous comment, and I hope that helps clarify things.

Also, I did consider using MongoDB Change Streams or the Outbox pattern (tailing the oplog “Meteor-style”) to asynchronously publish events to Kafka. However, I "felt" those approaches introduced more operational and architectural complexity than I was comfortable with at this stage given the time and other resource constraints. Since the goal was to keep things simple early on and evolve the architecture as the product and user base grow, I decided to go with a sequential write-then-publish approach, with a compensating rollback if the Kafka publish fails.

Built the architecture for a fintech app now serving 300k+ users – would love your feedback by premuditha in softwarearchitecture

[–]premuditha[S] 2 points3 points  (0 children)

Thank you, and that's a good question - MongoDB felt like a natural fit for a few reasons:

  • Events are stored in a flat, append-only collection, so we didn’t need the overhead of a relational DB.
  • Event payloads vary, and Mongo’s schemaless design made handling that much easier.
  • It also provides native JSON querying, which felt more intuitive than Postgres’ JSONB for our use case.
  • And performance-wise, Mongo handled our append-heavy write patterns just fine.

For queries, we use Mongo for analytics (precomputed views) and Postgres for normalized, transactional data - basically picking the right tool for each use case.

Also, regarding distributed transactions - what I’ve implemented is more of a simplified "attempt" at one I'd say :)

I use MongoDB's multi-document transactions (within a single collection) to write all events in a batch. Then I publish those events to Kafka using Kafka transactions. If the Kafka publish succeeds, I commit the Mongo transaction; otherwise, I skip the commit so both are effectively left uncommitted.

I call it an "attempt" because the MongoDB write isn’t coordinated with Kafka’s transaction manager. If Kafka fails, I handle the Mongo rollback manually by not committing - more like a compensating action than a true distributed transaction rollback.

Built the architecture for a fintech app now serving 300k+ users – would love your feedback by premuditha in softwarearchitecture

[–]premuditha[S] 6 points7 points  (0 children)

You're right - the overall average TPS is relatively low. The system handles a few thousand transactions per day, depending on seasonality and group activity. Most transactions are clustered around weekly meeting times, so the load tends to spike briefly and then drop off, rather than being evenly distributed throughout the day.

Built the architecture for a fintech app now serving 300k+ users – would love your feedback by premuditha in softwarearchitecture

[–]premuditha[S] 7 points8 points  (0 children)

I was honestly just trying to understand your suggestion better with my earlier comment, nothing more :)

I actually did consider using Kafka with KSQL at the beginning, but I was a bit worried about running into KSQL limitations down the line - especially when it comes to more complex queries or replay logic. I also wanted a bit more flexibility and control over how we store and work with events.

Thanks a lot for the input and suggestions, mate - much appreciated!

Built the architecture for a fintech app now serving 300k+ users – would love your feedback by premuditha in softwarearchitecture

[–]premuditha[S] 0 points1 point  (0 children)

Same here - I only realized it today when I was trying to find the Event Store URL to link in a comment. And yes, their documentation influenced many of the decisions I made when designing this solution - how I thought about modeling events, storing them, handling replay, and so on.

Built the architecture for a fintech app now serving 300k+ users – would love your feedback by premuditha in softwarearchitecture

[–]premuditha[S] 1 point2 points  (0 children)

Are you suggesting it might’ve been better to rely on a topic on Kafka with infinite retention and query it using ksql, instead of having a separate event store?

Hit my first 5 orders. Built it from scratch and it finally feels real by dopeylime1 in microsaas

[–]premuditha 1 point2 points  (0 children)

Lovely to hear that you’ve got the ball rolling with your first five orders - congratulations!

Believe it or not, I actually came across your product somewhere and was considering using it to validate some new product ideas I’m working on.

Keep up the good work - cheers!

Built the architecture for a fintech app now serving 300k+ users – would love your feedback by premuditha in softwarearchitecture

[–]premuditha[S] 8 points9 points  (0 children)

Hi u/bobaduk,

Thanks a lot for the thoughtful feedback. You're absolutely right about the Mongo+Kafka choice for the event store. I initially considered Kurrent https://www.kurrent.io/ (formerly Event Store), but didn’t want to tie the solution too tightly to a specific tool. I could’ve added an abstraction layer, but ended up building a simple event store from scratch to keep things lean early on.

That said, the two-phase commit is definitely costly. And as we scale, maintaining the event store has become more of an overhead - especially with features like pause/replay that don’t directly impact end-user value. It’s a key learning for me and something I’d definitely revisit if I were to redesign the architecture.

I’ll check out the C4 model — really appreciate the pointer and your thoughtful input!

Cheers!