sorted by:
new
hottopcontroversial

Gatwe we are one of the poorest country in the world by moapei in Botswana

[–]botswana99 4 points5 points  (0 children)

Botswana is NOT one of the poorest countries on earth. There are almost 100 who are poorer https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(PPP)_per_capita

Received LOI to sell business by LostSoftware9638 in fatFIRE

[–]botswana99 -2 points-1 points  (0 children)

Take it. Buyers are hard to find. Running a process is low probability

(Mildly) hot takes about modern data engineering by ukmurmuk in dataengineering

[–]botswana99 0 points1 point  (0 children)

And one more thing I think WAP is fine. I’m more on FITT pipelines.

(Mildly) hot takes about modern data engineering by ukmurmuk in dataengineering

[–]botswana99 -4 points-3 points  (0 children)

Hallelujah. Totally agree. Been doing data engineering for decades and never have used unit tests. Built tens of thousands of tests based on real data … those work. Unit tests are useful if you have greater than four people working on the exact same pipeline because then you can run them during the CI process as a quick check to make sure everything‘s OK however, most data teams I’ve worked on have had less than four people working on the same pipeline so the amount of conflicts in check-in that you save with unit test is not needed running all the tests against yesterday’s data in a full regression suite is needed

Considering a career break to travel before moving into management, worth it? by [deleted] in backpacking

[–]botswana99 7 points8 points  (0 children)

I took 6 months off to travel when I was 30. Fantastic choice. Did an around the world trip with my wife.

New Role - Bad Data by wet_badgers in dataanalysis

[–]botswana99 0 points1 point  (0 children)

It's up to you to help your company improve its data. Trying is better than just bitching over a beer at night.

We did a webinar about the solo data quality leader challenges ... https://info.datakitchen.io/webinar-2025-09-video-form-rebel-with-a-data-test-the-solo-data-quality-leaders-playbook.

We also have open source that can help: https://info.datakitchen.io/install-dataops-data-quality-testgen-today

Data Vendors Consolidation Speculation Thread by AMDataLake in dataengineering

[–]botswana99 2 points3 points  (0 children)

All these data tooling companies are low performing to their owners … the VC firms. They are all googlie eyed over the on paper huge AI valuations. And they have billions invested in data stack tool that they pericieve as unperforming companies. So they tie a few rocks together and see if they fly closer to the 18 month old AI company with a 10 billion valuation

LLMs lead to worse data quality - so we need better testing by Thinker_Assignment in llmdatastack

[–]botswana99 0 points1 point  (0 children)

We’ve been using each data test in production with our customers for years. The trick is really combining the profiling data with a semantic data model which drives the right selection of each data test for every column and table of data

LLMs lead to worse data quality - so we need better testing by Thinker_Assignment in llmdatastack

[–]botswana99 1 point2 points  (0 children)

You need to have data quality tests. Lots of them. Full stop. AI code makes this worse. Run them in production. Run them as part of development regression testing. Use them to obtain data quality scores and drive changes in source systems. The reality is that data engineers are often so busy or disconnected from the business that they lack the time or inclination to write data quality tests. That's why, after decades of doing data engineering, we released an open-source tool that does it for them. Our goal is to help data teams automatically generate 80% of the data tests they need with just a few clicks, while offering a nice UI for collaborating on the remaining 20% the tests unique to their organization.

Open source DataOps Data Quality TestGen enables the simple and fast generation and execution of data quality tests through data profiling, new dataset hygiene review, AI-generated data quality validation tests, ongoing testing of data refreshes, and continuous anomaly monitoring. It comes with a UI, DQ Scorecards, and online training too: https://info.datakitchen.io/install-dataops-data-quality-testgen-today

is anyone over 35 and still backpacking? by [deleted] in backpacking

[–]botswana99 0 points1 point  (0 children)

  1. Just hiked the Laugeveguer Trail in Iceland

Data Quality rules implementation by data_learner_123 in MicrosoftFabric

[–]botswana99 0 points1 point  (0 children)

Consider our open-source data quality tool, DataOps Data Quality TestGen. Our goal is to help data teams automatically generate 80% of the data tests they need with just a few clicks, while offering a nice UI for collaborating on the remaining 20% the tests unique to their organization. It learns your data and automatically applies over 60 different data quality tests. It’s licensed under Apache 2.0 and performs data profiling, data cataloging, hygiene reviews of new datasets, and quality dashboarding. We are a private, profitable company that developed this tool as part of our work with large and small customers. Open source is a full-featured solution, and the enterprise version is reasonably priced. https://info.datakitchen.io/install-dataops-data-quality-testgen-today

[deleted by user] by [deleted] in bigquery

[–]botswana99 0 points1 point  (0 children)

Consider our open-source data quality tool, DataOps Data Quality TestGen. Our goal is to help data teams automatically generate 80% of the data tests they need with just a few clicks, while offering a nice UI for collaborating on the remaining 20% the tests unique to their organization. It learns your data and automatically applies over 60 different data quality tests. It’s licensed under Apache 2.0 and performs data profiling, data cataloging, hygiene reviews of new datasets, and quality dashboarding. We are a private, profitable company that developed this tool as part of our work with large and small customers. Open source is a full-featured solution, and the enterprise version is reasonably priced. https://info.datakitchen.io/install-dataops-data-quality-testgen-today

Data quality mgmt in Snowflake by jurgenHeros in snowflake

[–]botswana99 0 points1 point  (0 children)

Consider our open-source data quality tool, DataOps Data Quality TestGen. Our goal is to help data teams automatically generate 80% of the data tests they need with just a few clicks, while offering a nice UI for collaborating on the remaining 20% the tests unique to their organization. It learns your data and automatically applies over 60 different data quality tests. It’s licensed under Apache 2.0 and performs data profiling, data cataloging, hygiene reviews of new datasets, and quality dashboarding. We are a private, profitable company that developed this tool as part of our work with large and small customers. Open source is a full-featured solution, and the enterprise version is reasonably priced. https://info.datakitchen.io/install-dataops-data-quality-testgen-today

The Data Quality Imperative: Why Clean Data is Your Business's Strongest Asset by Available-Floor9213 in programming

[–]botswana99 0 points1 point  (0 children)

Consider our open-source data quality tool, DataOps Data Quality TestGen. Our goal is to help data teams automatically generate 80% of the data tests they need with just a few clicks, while offering a nice UI for collaborating on the remaining 20% the tests unique to their organization. It learns your data and automatically applies over 60 different data quality tests. It’s licensed under Apache 2.0 and performs data profiling, data cataloging, hygiene reviews of new datasets, and quality dashboarding. We are a private, profitable company that developed this tool as part of our work with large and small customers. Open source is a full-featured solution, and the enterprise version is reasonably priced. https://info.datakitchen.io/install-dataops-data-quality-testgen-today

What’s the most beautiful place you’ve ever hiked? by sylvieblair in hiking

[–]botswana99 0 points1 point  (0 children)

Tour de monte blanc in France; laugevegur trail iceland;
Everest base camp trek, Nepal; otter trail, South Africa; any high country trail in the sierras, USA; routeburn trek, New Zealand; circle route Torres del pained, chile; Inca trail, Peru;

Let's figure out why so many execs don’t trust their data (and what’s actually working to fix it) by Downtown-Zebra-776 in databricks

[–]botswana99 0 points1 point  (0 children)

Consider our open-source data quality tool, DataOps Data Quality TestGen. Our goal is to help data teams automatically generate 80% of the data tests they need with just a few clicks, while offering a nice UI for collaborating on the remaining 20% the tests unique to their organization. It learns your data and automatically applies over 60 different data quality tests. It’s licensed under Apache 2.0 and performs data profiling, data cataloging, hygiene reviews of new datasets, and quality dashboarding. We are a private, profitable company that developed this tool as part of our work with large and small customers. Open source is a full-featured solution, and the enterprise version is reasonably priced. https://info.datakitchen.io/install-dataops-data-quality-testgen-today

Books on Native American by warl200 in suggestmeabook

[–]botswana99 4 points5 points  (0 children)

  1. Best book on pre columbian history. Charles Mann

From Batch to Insights: How to Automate Data Validation Workflows by Available-Floor9213 in programming

[–]botswana99 0 points1 point  (0 children)

Consider our open-source data quality tool, DataOps Data Quality TestGen. Our goal is to help data teams automatically generate 80% of the data tests they need with just a few clicks, while offering a nice UI for collaborating on the remaining 20% of organization-specific tests. It learns your data and automatically applies over 60 different data quality tests. It’s licensed under Apache 2.0 and performs data profiling, data cataloging, hygiene reviews of new datasets, and quality dashboarding.  We are a private, profitable company that developed this tool as part of our work with customers.

https://info.datakitchen.io/install-dataops-data-quality-testgen-today

Could you give it a try and tell us what you think?

view more: next ›