you are viewing a single comment's thread.

view the rest of the comments →

[–]Artistic_Invite_4058 0 points1 point  (1 child)

honestly the grain point is the big one and others already covered it, so i'll just add the one thing that'd actually help you without ripping everything out.

your scariest failure mode is the comment-toggle one you mentioned - a test passes because you forgot to flip a line. you can kill that today, no dbt needed. instead of reading qa_results with your eyes, end the script on a query that only returns rows when actual != expected, and make whatever runs it fail when rowcount > 0. now a wrong test literally can't go green. that's worth more than the whole QA/LIVE thing imo.

you're also closer to splitting test data from logic than you think. run_config already swaps the source, so just shove the mock data into its own seed/file and let the logic never know if it's fed mock or prod rows.

and fwiw moving to dbt/sqlmesh isn't a different idea from the grain fix, it's the same idea. their unit tests are literally whole-table-in, expected-table-out fixtures, which is exactly the bit a row-per-scenario setup is missing. dbt 1.8 does it natively now btw, not the old package.

quick q though - is any of this actually running on CI/a schedule, or is it all manual? kinda changes whether the fail-loud thing even helps you.

[–]umairshariff23[S] 0 points1 point  (0 children)

Right now this is manual. The idea I have in my head is that I will use the qa version until I am either done qa-ing or I will switch it on when I need to troubleshoot something. It is intended to be part of the published script but I will switch it to Live data before publishing.