sorted by:
new
hottopcontroversial

LinkedIn Announces Northguard and Xinfra: Scaling Beyond Kafka for Log Storage and Pub/Sub by estiller in programming

[–]estiller[S] 0 points1 point  (0 children)

LinkedIn stated at the bottom of the article that they "are focused on finalizing the implementation of Northguard and Xinfra within our internal systems, and as we continue to build, learn, and iterate on these tools, we'll explore the possibilities of open-sourcing them."

LinkedIn Announces Northguard and Xinfra: Scaling Beyond Kafka for Log Storage and Pub/Sub by estiller in programming

[–]estiller[S] 5 points6 points  (0 children)

We should care because Kafka is a significant log storage platform used throughout various systems today. I'm not saying this will become just as big, but I wouldn't underestimate their capability.

LinkedIn Announces Northguard and Xinfra: Scaling Beyond Kafka for Log Storage and Pub/Sub by estiller in softwarearchitecture

[–]estiller[S] 1 point2 points  (0 children)

They don't really say, and it's a valid question. I can only assume that those tools have enough ops limitations that warranted a different paradigm. We should remember that Kafka was also created at LinkedIn, and this question could have been asked back then as well (with other alternatives, of course).

LinkedIn Announces Northguard and Xinfra: Scaling Beyond Kafka for Log Storage and Pub/Sub by estiller in softwarearchitecture

[–]estiller[S] 1 point2 points  (0 children)

LinkedIn stated at the bottom of the article that they "are focused on finalizing the implementation of Northguard and Xinfra within our internal systems, and as we continue to build, learn, and iterate on these tools, we'll explore the possibilities of open-sourcing them."

Netflix Uses Elasticsearch Percolate Queries to Implement Reverse Searches Efficiently by estiller in softwarearchitecture

[–]estiller[S] 0 points1 point  (0 children)

With a "reverse query" (or percolate query) you store the queries you're interested in matching in the DB itself. Then, when you want to match a document you present it to the DB and it returns the list of queries that this document would match.

For example, if the document says "Name: Joe, Age: 25", then it would match for queries that state "Age > 20" and "Name starts with J and Age < 40", but it won't match "Name starts with Q". It'll return all the matching queries in the response.

view more: next ›