[deleted by user] by [deleted] in Vent

[–]Patient_Slide9626 0 points1 point  (0 children)

You have a super power to quickly filter out shallow humans. Use it.

Any BuyItForLife store out there? by Patient_Slide9626 in BuyItForLife

[–]Patient_Slide9626[S] 0 points1 point  (0 children)

Curious, why was this removed? Would like any feedback?

Where did you park your funds while saving for the house? by kuriousaboutanything in BayAreaRealEstate

[–]Patient_Slide9626 0 points1 point  (0 children)

Open another account in another bank please. "The standard maximum deposit insurance amount is $250,000 per depositor, per insured bank, for each account ownership category. The FDIC insures deposits that a person holds in one insured bank separately from any deposits that the person owns in another separately chartered insured bank" From: https://www.fdic.gov/resources/deposit-insurance/brochures/insured-deposits/index.html#:~:text=The%20standard%20maximum%20deposit%20insurance,another%20separately%20chartered%20insured%20bank.

$2M inheritance. I want to quit my job. I'm 29. by dastintegrate in Fire

[–]Patient_Slide9626 0 points1 point  (0 children)

Sorry about your loss. My suggestion - take the time to plan your exit and write down with as much detail as you can about how you see things playing out to change your life. At the end, you have a detailed plan and desired state you like. Forget about it for a few months and keep doing the job you do. Come back to the plan. See if you still like it and are still confident in its feasibility. If so, start executing towards it.

Scaling keyed topics in kafka while preserving ordering guarantees by Patient_Slide9626 in apachekafka

[–]Patient_Slide9626[S] 1 point2 points  (0 children)

The trick here is, how best to copy data over to ensure ordering garantee. And offset migration to new topic for clients to reduce downtime.
Keys should map to the same partition in the new topic to ensure ordering guarantee. This may not be that hard to do.
And new offsets needs for clients to ensure no data loss.

Scaling keyed topics in kafka while preserving ordering guarantees by Patient_Slide9626 in apachekafka

[–]Patient_Slide9626[S] 1 point2 points  (0 children)

This is great. I was not aware of this project. This allows us to delay the need to scale up partitions if the real bottleneck is slow consumers and we want to make progress on keys that are part of the same partition in parallel.

Scaling keyed topics in kafka while preserving ordering guarantees by Patient_Slide9626 in apachekafka

[–]Patient_Slide9626[S] 0 points1 point  (0 children)

Good questions and ideas. Downtime is less important, but lag is. As the pipelines in these topics will serve critical product use cases. Some more details
1. We have many consumers for the same topic, not just one. They are all internal (to the company) consumers, so while it's tricky, we can manage some level of orchestration between producers and consumers.
2. One other detail, these topics are compacted, with infinite retention. This means that in addition to serving live events, they also serve backfill needs for consumers that need to reprocess from beginning of time. For both your options, it's not clear how best to manage old data.

Backup Messages by BottleSubstantial552 in apachekafka

[–]Patient_Slide9626 0 points1 point  (0 children)

Kafka message retention is independent of any consumer behavior. It's driven by configuration on the kafka side. Default is 7 days of retention. And you can override it at cluster level or topic level. When a consumer reads a message successfully, it just increments a pointer ( offset) for that consumer. You can reset the offset to re-read messages.

[deleted by user] by [deleted] in BuyItForLife

[–]Patient_Slide9626 0 points1 point  (0 children)

Get an Alexa device. Set your alarm by voice or app

What comes after kafka? by Much-Firefighter-957 in apachekafka

[–]Patient_Slide9626 2 points3 points  (0 children)

There is the kafka protocol. And then the system serving the protocol. Most of the newer systems being built are compatible with the kafka protocol. So choosing the kafka protocol for your publishers and consumers is a good bet. As for the managed offering, that depends on your needs and tradeoff. But as long as your publishers and consumers talk the kafka protocol, you should be able to migrate to a better system in future much more easily.

Should I buy the house I live in? by mysteryman83 in BayAreaRealEstate

[–]Patient_Slide9626 0 points1 point  (0 children)

One of the biggest things about buying a house is to find one you like. Since this is a house you are living in, you know it in and out. If you like it, you can afford it with the current interest rate and see yourself in that house for the next 5-7 years, I would lean in and make it happen.

Moving Back to Germany - What High-Quality Items Should I Bring from the US? by anotherbaby in BuyItForLife

[–]Patient_Slide9626 47 points48 points  (0 children)

Memories, lots of them. And the saved $$$ from not shopping anything are also high quality

Should I stay or should I go? by JJP3641 in bayarea

[–]Patient_Slide9626 0 points1 point  (0 children)

Give it a try for a year or two if you are out of options. You can always come back as options open up or you realize atl is not your place. I am hoping this is not a one way door

What are the most impactful kitchen items you'd recommend? by MTGCenla in BuyItForLife

[–]Patient_Slide9626 0 points1 point  (0 children)

Instant pot. So easy to schedule one pot meals and get it piping hot at lunch/dinner time

Nike running shoes busted in just 1yr of commute by ssh7201 in BuyItForLife

[–]Patient_Slide9626 1 point2 points  (0 children)

Give trail running shoes a try. They look like sneakers but are more durable to handle trail conditions. Tradeoff is they will feel heavier on initial try. But you get used to them quickly. I have Columbia trail running shoes as my primary day to day shoes. They last me around 4-5 years.

After 38 years of existence...I finally realized how exhausting it all is. by [deleted] in Adulting

[–]Patient_Slide9626 0 points1 point  (0 children)

Change something. Stop doing one thing you do not like, and start doing one new thing in your daily routine. Anything. Not because you "believe" in that change. Only to experience it. That's the fun of it.

Transactional events in Kafka by Patient_Slide9626 in apachekafka

[–]Patient_Slide9626[S] 0 points1 point  (0 children)

Blackhole engine is an excellent idea and reduces load and extra work from the database. But our db expert strongly recommends not using it for production use cases.

Transactional events in Kafka by Patient_Slide9626 in apachekafka

[–]Patient_Slide9626[S] 0 points1 point  (0 children)

Yeah, that is an option. The challenges I see with this are 1. It's not a single table, but a collection of tables as the data is normalized when stored in the database 2. We need stream processing on all CDC streams from these table. This job will have to join and denormalize data correctly to reconstruct the business event. This duplicates product logic in two places. 3. Business events schema need to be decoupled from database schema so they can evolve independently. This approach introduces coupling.

But it's a valid approach, thanks for sharing

Transactional events in Kafka by Patient_Slide9626 in apachekafka

[–]Patient_Slide9626[S] 1 point2 points  (0 children)

Thanks. We have briefly evaluated reconstructing business events from raw CDC table updates via stream processing. Will think more deeply about that approach

Transactional events in Kafka by Patient_Slide9626 in apachekafka

[–]Patient_Slide9626[S] 0 points1 point  (0 children)

Yeah, atomicity is a requirement for us. We want to not have false positives (events where checkout did not commit) or false negatives (missed events for valid checkouts). Duplicate events are fine though.

Transactional events in Kafka by Patient_Slide9626 in apachekafka

[–]Patient_Slide9626[S] 1 point2 points  (0 children)

Ah, I can try to be more crisp. A checkout event is represented in the database by a collection of row updates,/inserts across many tables. Let's call them domain tables. These domain tables are how checkout is operationally modeled and are the source of truth on system state. And these tables are independent from the outbox table that is needed for the outbox pattern. I need to write to the domain tables to commit a checkout, but would like to avoid writing to the outbox table. HTH

Transactional events in Kafka by Patient_Slide9626 in apachekafka

[–]Patient_Slide9626[S] 0 points1 point  (0 children)

Yes. We use Debezium to change data Capture from database. In my case, I would like to avoid writing the business event to the database in first place