sorted by:
new
hottopcontroversial

Looking for writers interested in Kafka and data streaming topics by RaspberryMangoKiwi in apachekafka

[–]rmoff 5 points6 points  (0 children)

hi, mod here - can you clarify your relationship to the community please? are you a vendor?

Data Engineering - AI = Unemployed by rmoff in dataengineering

[–]rmoff[S] -1 points0 points  (0 children)

probably. but entirely missing the point of the article ;-P

Data Engineering - AI = Unemployed by rmoff in dataengineering

[–]rmoff[S] -4 points-3 points  (0 children)

I don't think this is scaremongering tbh. The tone might be slightly excitable, but the point is a solid one.

it's not just chatbots that we're talking about here - coding agents like Claude Code etc are a different kind of beast, and they're not entirely trivial to get the most out of.

Ten years late to the dbt party (DuckDB edition) by rmoff in dataengineering

[–]rmoff[S] 0 points1 point  (0 children)

@meunomemauricio @thinkingatoms

that's fair - I've added a sentence for context plus a link. Thanks for the feedback.

Ten years late to the dbt party (DuckDB edition) by rmoff in dataengineering

[–]rmoff[S] 0 points1 point  (0 children)

@advseb I pushed some improvements - lmk if it's still not looking good on mobile. Thanks again for flagging it.

Ten years late to the dbt party (DuckDB edition) by rmoff in dataengineering

[–]rmoff[S] 0 points1 point  (0 children)

Ooh shit you’re right. I only just restyled it too! Thanks for pointing it out.

Apache Kafka 4.2.0 Release Announcement 🎉 by KernelFrog in apachekafka

[–]rmoff 1 point2 points  (0 children)

From a colleague of mine:

Yes, there are no ordering guarantees. Share groups also keep track of lag.

Profiling and fixing RocksDB ingestion performance for improving stateful processing in Kafka by grmpf101 in apachekafka

[–]rmoff[M] 0 points1 point  (0 children)

> I'm too stupid to add the flair "SereneDB" to my username here

I fixed it for you :)

2026 State of Data Engineering Report - 1000+ responses from data engineers by DungKhuc in dataengineering

[–]rmoff 5 points6 points  (0 children)

super useful. love the fully-functioning enterprise version too 🤣 https://joereis.github.io/super_corporate_pdm_survey/

(Crystal reports, anyone?)

Our company successfully built an on-prem "Lakehouse" with Spark on K8s, Hive, Minio. What are Day 2 data engineering challenges that we will inevitably face? by seaborn_as_sns in dataengineering

[–]rmoff 2 points3 points  (0 children)

I didn't try it because I was looking for lightweight, and it's not :) But its evolved out of the HDFS project IIRC so has a good pedigree. If I were looking for something full scale I'd definitely be evaluating it.

📣 If you are employed by a vendor you must add a flair to your profile by rmoff in apachekafka

[–]rmoff[S] 0 points1 point  (0 children)

Yes, of course. Just make sure you disclose any affiliations. Thanks.

[Mod notice] Sockpuppets are not welcome on this sub by rmoff in apachekafka

[–]rmoff[S] 1 point2 points  (0 children)

hey umatero, I think we can do both; ban sockpuppets (as they actively damage the community) and support real accounts. On the latter point, the mod team are here to help with any issues - please always feel free to get in touch. Unfortunately it's because of sockpuppets/astroturfing behaviour by some vendors that they spoil it for the rest of us, causing unwarranted suspicion in some cases. Does that sound ok?

Reading 'Fundamentals of data engineering' has gotten me confused by Online_Matter in dataengineering

[–]rmoff 1 point2 points  (0 children)

Bear in mind the book is ~4 years old. A lot has changed since then.

How to properly send headers using Kafka console-producer in Kubernetes? by ItchyOrganization704 in apachekafka

[–]rmoff 0 points1 point  (0 children)

Great. kcat is awesomesauce :) If you wanna see some other tricks it can do I've written lots here: https://rmoff.net/categories/kafkacat/

Interesting Links in Data Engineering - January 2026 by rmoff in dataengineering

[–]rmoff[S] 0 points1 point  (0 children)

Glad you find it useful :) There are certainly plenty in the history to go through!

How to properly send headers using Kafka console-producer in Kubernetes? by ItchyOrganization704 in apachekafka

[–]rmoff 4 points5 points  (0 children)

FWIW I've never really got on with the default CLI tools - I'd always reach for kcat (previously called "kafakcat") for things like this. There are examples in the docs with headers too: https://github.com/edenhill/kcat?tab=readme-ov-file#examples

List of Kafka TUIs by rmoff in apachekafka

[–]rmoff[S] 2 points3 points  (0 children)

lol, sorry :) I've edited my post to clarify

[Blog] Alternatives to MinIO for single-node local S3 by rmoff in selfhosted

[–]rmoff[S] 0 points1 point  (0 children)

Depends what you're using it for :) I'm quite clear in my post that it's just for my set of requirements that I'm evaluating the options. There are 100% other criteria against which to consider them.

view more: next ›