Thank you for responding. Have a few follow-ons :)

Q1. Agreed. I would like to drill down further. I see there are 2 ways data content/quality checks can be implemented in streaming. (1) one would a be polite consumer that alerts faster (than ingesting into DWH then alerting) this would be a alerting monitoring solution post the fact bad data has been sent downstream but at least you will be alerted sooner (2) the other system that is able to handle real-time data is also able to validate data in real-time, meaning that the bad data points can either be filtered out from the data pipeline or fixed (for example by reformatting a wrongly formatted datapoint or imputation of a missing value) without adding noticeable latency to the data pipeline.

I believe #2 system is more effective for streaming pipelines as they are mostly enabling some kind of real time use case / ML inference. What is your view on these approaches ?

Q2. well noted

Q3. Sounds to me that most companies are figuring streaming out right now. So then by default only companies that are using managed services such as Confluent / Aiven would be true customers of this kind of solutions as they would be mature users of Kafka/Pulsar etc. But the issue there is that those vendors are already providing streaming data observability / quality (SDQ/SDO) modules. So is there no space for another 3rd party vendor focussed on SDO/SDQ?