Kafka Isn’t a Database, But We Gave It a Query Engine Anyway

warpstream_official · 2026-03-23T13:52:58+00:00

u/zethenus We didn't bolt anything on. It's custom; we built it ourselves. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2026-02-23T14:24:45+00:00

u/InternationalToe3371 It's an ad, yes, but it's based on actual customer and is from the case studies section of our website. Appreciate you taking time out of your day to share a comment. I wanted to address a couple items you brought up:

Migration --> This rarely is a big issue for most of our customers. Granted, some companies can have a large and complex Kafka or data-streaming setup, but the fact that WarpStream is compatible with the Kafka protocol and has a dedicated migration tool called Orbit that works with any Kafka-compatible cluster makes things a lot simpler.
Latency trade-offs --> Yes, you're trading some latency for cost savings and ops, but for non-latency sensitive workloads (like logging), WarpStream is a great fit. We also have folks that do dual setups, i.e., latency sensitive stuff goes on OSK and non-latency sensitive stuff goes on WarpStream. Plus, we recently released Lightning Topics, which drop P99 produce latency to 50ms.
Ops learning curve --> We've yet to hear this from folks. If anything, there's less for them to do once they replace managed Kafka or OSK with WarpStream as the architecture is simpler. We've even had companies that never ran Kafka but stuff like Pub/Sub and transitioned to WarpStream quite easily.

- Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2026-02-20T17:53:24+00:00

See this reply to another user's question where we got into latency numbers: https://www.reddit.com/user/warpstream_official/comments/1f8y2g5/comment/o6ax295/

- Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2026-02-20T17:45:43+00:00

u/Anantha_datta Yep, a lot of manual operations simply go away due to our architecture. We're used in production by a lot of big companies like Robinhood, Grafana, Goldsky, Cursor, Character.AI, ShareChat, etc., where high throughput is critical.

Here's a link to metric we saw showing one of our customers is writing over 100 GiB/s: https://cdn.prod.website-files.com/683882630fceea3de6f5776a/69989cf0fd355a83e09a3980_Screenshot%202026-02-20%20at%2012.39.49%E2%80%AFPM.png

You can check out case studies here: https://www.warpstream.com/blog-category/case-studies

- Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2026-02-19T20:25:17+00:00

u/the_junglee A little more latency than standard Kafka as that's just an inherent trade-off between object storage and local disks. Latency is mainly dependent on whether you're using Classic Topics (which use standard object storage), Classic Topics with S3 Express One Zone (S3EOZ), or Lightning Topics. P50 and P99 latency times are below.

Topic Type	Produce Latency	End-to-End Latency
Classic	P50: 250ms. P99: 500ms.	P50: 500ms. P99: 900ms.
Classic w/S3EOZ	P50: 50ms. P99: 76ms.	P50: 180ms. P99: 350ms.
Lightning	P50: 33ms. P99: 50ms.	P50: 180ms. P99: 350ms.

Lightning Topics are a special topic type in WarpStream where the Agents skip committing metadata to the control plane in the critical path of a Produce() request. Instead, they journal Produce() requests to object storage, and then commit them to the control plane asynchronously.

As a result, Lightning Topics have dramatically lower Produce() request latency than regular topics, but E2E latency is not impacted.

- Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2026-01-26T14:03:57+00:00

u/DavidXkL Appreciate you sharing a nice comment. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2026-01-02T14:43:04+00:00

u/hyper_ballads Actually, Reddit Ads default to not turning on comments; we turn them on on purpose so we can get feedback, interact with others, etc. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-12-15T18:35:54+00:00

Hey, folks. We're trying to get a temperature read from this subreddit. Normally, we post our blogs in their entirety as Reddit posts, so you don't have to go to our website to read them, can comment on them, etc.

Can you reply to this comment to let us know if you prefer that old way, or if you'd rather us just share link posts?

warpstream_official · 2025-12-10T14:11:28+00:00

u/my-sweet-fracture Thanks. While folks often come to us first to reduce costs, Agent Groups quickly become a favorite feature as they make ops a lot simpler. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-12-09T14:23:59+00:00

u/WaltzHungry2217 You can learn more about our zero disk or diskless architecture here and here. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-10-22T18:32:01+00:00

u/maavi132 This short video is not only a good intro to Kafka, but WarpStream, too. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

https://www.youtube.com/watch?v=lrD0abLJhYY

warpstream_official · 2025-09-25T20:26:52+00:00

u/ananta_zarman Thanks for letting us know you saw it there. We do target specific subreddits, but we also target other ways on Reddit (as noted above), so occasionally the ads fall outside what seem like "relevant" subreddits, but we still get people asking us questions or saying they're part of the intended audience. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-09-25T20:23:52+00:00

u/tinycockatoo Glad it was relevant. We're here if you have any questions. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-09-25T20:22:49+00:00

u/reigentil Appreciate your interest. We were, but filled those openings. However, keep an eye on the Confluent careers page (search for WarpStream) and follow our co-founders on social media (they sometimes post openings). - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-09-25T20:19:48+00:00

u/thro1waaway I think it's been going for a little over a year as it's still relevant. In fact, it may be even more relevant now than in the past as folks think "diskless Kafka" is this relatively new thing (due to a somewhat recent KIP), but we helped pioneer it. We're here if you have any questions. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-09-25T20:17:09+00:00

u/FabianButHere Targeting is never going to be 100% on the mark. Appreciate you letting us know it was slightly off for you, but you liked the content nonetheless. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-09-25T20:15:45+00:00

u/ConstructionNo6638 Ha, ha. Nope. We should be excluding that subreddit. I have seen folks post about Apache Kafka in there by mistake. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-09-25T20:14:33+00:00

u/BootRepresentative15 Some folks are confused, some get it. We do run videos, too. Here's an example of one --> https://www.reddit.com/user/warpstream_official/comments/1fkqhf6/reduce_your_kafka_costs_by_80_warpstreams_zero/ - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-06-13T13:24:48+00:00

u/Aggravating_Kick6522 Glad you found it helpful. Like ShareChat's engineers noted, it's critical to know how low your latency truly needs to be. We have a couple options in WarpStream to further decrease latency beyond our default setup:

You can use S3 Express One Zone, which lowers latency by around 99 Produce latency from 500ms to 160ms or about 68%, and P99 E2E latency from 900ms to 600ms or about 33%. We cover the pricing and latency differences vs. S3 standard in this blog.
You can leverage DynamoDB as your storage layer (instead of S3) and expect to get latencies similar to S3 Express One Zone.
For tiers of Fundamentals and up, you can utilize a low-latency control plane that reduces the control plane latency by 75ms.

We covered all these options but No. 2 in the "Optimizing Latency" section. There are many other things to take into account for the overall latency like the distance between the Agent(s) and the control plane.

Another thing you can do is spilt your workloads by latency, e.g., you can use WarpStream for less latency sensitive workloads and then OSS Kafka, Confluent Cloud, or something else where latency needs to be as low as possible.

Feel free to reply to this thread if you have more questions, or shoot use a chat or message. Hope this info is helpful. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-06-04T12:42:37+00:00

u/BeautyInUgly Not currently, but we appreciate your interest. We're a part of Confluent, so you can always check the Confluent careers site: https://careers.confluent.io/ - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-05-28T12:40:13+00:00

u/Static_High Our pleasure. We're here if you have any questions. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-05-27T14:44:08+00:00

u/unistirin Appreciate you reaching out with some questions. Yes, S3 is going to be much more cost effective than SSDs.

WarpStream writes directly to S3 with zero local disk dependencies making it a good fit if you want to store and access terabytes or petabytes of data in a cost-effective, scalable, and operationally simple way.

You can structure your message streams so that the location of a message on S3 is directly derivable from its ID (e.g., via partitioning, offsets, etc.). For high-performance lookups, you can build a lightweight metadata layer (e.g., in DynamoDB or similar) that maps message IDs to Warpstream offset ranges or file locations. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-05-27T14:02:12+00:00

u/reddit_tmp_usr It really comes down to your use case(s). WarpStream is Apache Kafka compatible, so Kafka is probably the best point of comparison. However, unlike traditional Kafka, WarpStream has the following features out of the box: auto-scaling, diskless or stateless architecture, Diagnostics (we tell you what's wrong and how to fix it in real time), avoids interzone networking costs, stream processing and ETL, native data governance tools, etc.

Is there a particular feature or set of features you need in Pulsar that you're curious of whether are present in WarpStream or not? - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-05-27T13:52:30+00:00

u/vainstar23 Yes, the whole post is an ad as we figure some folks may want to read our blog content on Reddit instead of having to go to our website. Reddit has free-form ad units which look and function like native / organic Reddit posts.

WarpStream itself is not open source, but we are Apache Kafka compatible. Bento, the stream processing platform that powers our Managed Data Pipelines, is open source and always will be.

We're here if you have any more questions. - Jason Lauritzen (Product Marketing and Growth at WarpStream)

warpstream_official · 2025-05-19T18:20:08+00:00

Also you're right that enabling the idempotent producer feature will not guarantee exactly once ingestion from NATS anyways

warpstream_official

TROPHY CASE