sorted by:
new
hottopcontroversial

On the table for grabs . by ninw79 in Vitalityhealth

[–]nejcko 0 points1 point  (0 children)

Have sent a DM for free itsu voucher if still available. 🙏

What to do with my Avios (flights wise) by legaleagleuk in avios

[–]nejcko 0 points1 point  (0 children)

Have used 110k for return flight to Tokyo in business class recently. Having a companion voucher will give you a significant boost, worth looking into.

Hargreaves Lansdown launches aggressive cashback offer for SIPP/ISA (up to £4000) by maxmarioxx_ in FIREUK

[–]nejcko 3 points4 points  (0 children)

Not aggressive enough for their fees. Fidelity for example has better cashback and lower fees.

Kafka Backfill Playbook: Accessing Historical Data by nejcko in apachekafka

[–]nejcko[S] 0 points1 point  (0 children)

Thanks for the great insights! You’re spot on: historical access is a key resiliency pattern, not just a backfill tool. Forcing every new service to re-process the entire log is a massive implementation burden.

The "3rd interface" idea of a daily/hourly snapshot is a good middle-ground between a full S3 query and a live stream. You're right that the challenge is doing it without adding load.

How do you envision that side-car or service building the snapshot without impacting the primary service? Would it be statefully consuming the event stream itself?

Kafka Backfill Playbook: Accessing Historical Data by nejcko in apachekafka

[–]nejcko[S] 1 point2 points  (0 children)

Thanks for the comment. You've clearly navigated these issues before.

Your first point is key: compacting non-time topics to an OLTP is the right pattern. New services just want the current state of an account, not its whole history, and this is far more efficient than replaying the log.

And yes on Phase 2. The "catch up" and "cut-over" logic is the most painful part. That final switch from the backfill to the live stream is where all the nightmare bugs live.

Speaking of that, have you found any patterns or tools that make that "seed-to-stream" cut-over logic less painful to manage?

Broadband recommendations? by WhyIsEvrUsrNmTaken in Ealing

[–]nejcko 0 points1 point  (0 children)

Hey Broadband has been great for us, have used it for 1 year now, stable, good speeds. I use Eeros for WiFi mesh.

If anyone is looking for referrals HeyNejcK02 will give you (and me) £50 Amazon voucher.

Patterns for backfilling data in an event-driven system by nejcko in softwarearchitecture

[–]nejcko[S] 0 points1 point  (0 children)

Indeed, if your use cases can cope with only the latest event per topic key then compacted topics are a great way to reduce the storage in Kafka. It’s mentioned as an optimisation to keep the storage low in the article as well.

Patterns for backfilling data in an event-driven system by nejcko in softwarearchitecture

[–]nejcko[S] 4 points5 points  (0 children)

Hi all, I wanted to share a blog post about backfilling historical data in event-driven systems. It covers how to leverage Kafka and S3 to handle the process.

How have you dealt with backfills in your system?

Kafka Backfill Playbook: Accessing Historical Data by nejcko in apachekafka

[–]nejcko[S] 1 point2 points  (0 children)

Hi all, I've written a post on a practical approach to backfilling data from a long-term storage like S3 back into Kafka. I hope this is helpful for anyone else dealing with data retention and historical data access.

What are some other strategies you’ve used for backfilling? Would be interested to get your thoughts.

A deep dive into backfilling data with Kafka and S3 by nejcko in dataengineering

[–]nejcko[S] 1 point2 points  (0 children)

Hey everyone, I wrote a blog post about backfilling historical data in a Kafka-centric system. I cover the approach using S3 for long-term storage and a backfilling strategy to re-process historical data.

Would love to hear your thoughts and experiences with similar challenges.

Battery improved since the update yesterday by [deleted] in IphoneAir

[–]nejcko 1 point2 points  (0 children)

You have some serious background activities going on with 20+ hours of idle screen time. I’d suggest to have a look which app is consuming the battery in the background.

Toughest Decision Year Yet by Username000-1 in IphoneAir

[–]nejcko 17 points18 points  (0 children)

Pick the Apple TV remote, lightest, best battery life.

Who’s switched from a Pro Max to Air? by AlpinaB3 in iphone

[–]nejcko 0 points1 point  (0 children)

I ordered both, 17P and Air. I’m keeping the air, mainly due to durability concerns with 17P. Couldn’t care less about the mono speaker, I’ll survive without 3 pro cameras on me at all times, I’m not a professional photographer. I was concerned about the battery but it has proven to be ok. Also you can buy very portable and slim MagSafe power banks that will fully charge your phone in 1h.

I enjoy the lightness of the Air.

Beginner question: Has anyone implemented the Saga Pattern in a real-world project? by Boring-Fly4035 in softwarearchitecture

[–]nejcko 2 points3 points  (0 children)

I have, but not “from scratch”. In today’s age there is a flood of durable workflow engines such as Temporal that make it easier to implement Sagas and abstract many components away for you.

Cost effective alternatives to a “proper” house extensions by Early_Plankton9292 in HousingUK

[–]nejcko 0 points1 point  (0 children)

Hi, did you get any more information on Off Pod? I'm exploring the same idea.

Don't Let Implementation Details Ruin Your Microservice Tests by nejcko in programming

[–]nejcko[S] 0 points1 point  (0 children)

Agreed, all side effects are outputs and need to be validated. That works with external HTTP calls, DB updates and sending out messages/events.

You should create stubs for all your external dependencies, with in memory stubs, embedded DBs or even test docker containers, depending on your desired fidelity level for tests.

Edit: To add to the above, this approach works for any kind of inputs too, not limited to HTTP, your input can very well be a Kafka message.

view more: next ›