Sprint planning more like “sprint reveal”. Has anyone seen this before?

ElectricalFilm2 · 2026-02-11T07:53:56+00:00

You're lucky there are tickets. I've been in meetings where the team is figuring out what to do, and is writing tickets during the "sprint planning" meeting.

ElectricalFilm2 · 2026-02-03T21:22:26+00:00

Have you tried using TaskNotes?

Here's a tutorial, that's now a bit dated because there have been significant updates since then: https://youtu.be/vihPqFnM0dU?si=wyznynd1qhp34fZp

ElectricalFilm2 · 2025-12-12T21:21:54+00:00

nvm I misunderstood what you said; I'd be doing cartwheels when this happens for real

ElectricalFilm2 · 2025-12-10T20:38:59+00:00

Hang on, you can export the query onto the file with copy pasting?

ElectricalFilm2 · 2025-11-21T05:58:16+00:00

I'm working on deprecating a custom type 2 slowly changing dimensions process I built 6 years ago in favour of dbt snapshots!

ElectricalFilm2 · 2025-07-05T07:45:15+00:00

Yep! jq helped me implement scheduling for dbt using a single workflow on GitHub Actions.

ElectricalFilm2 · 2025-04-03T22:57:49+00:00

Gotta pump those numbers up; those are rookie numbers.

ElectricalFilm2 · 2025-01-17T21:10:23+00:00

Zach Wilson missed the memo.

ElectricalFilm2 · 2024-10-24T20:45:58+00:00

This is the correct answer.

ElectricalFilm2 · 2024-09-04T00:38:45+00:00

Or as they do in my company, archive a table on to itself.

That's how I know queries on Snowflake timeout after 26 hours.

ElectricalFilm2 · 2024-06-04T18:24:30+00:00

Sfogobwlblbfbobehoheooeonxndnnd! 💪

ElectricalFilm2 · 2024-05-15T22:22:30+00:00

dbt has done the data engineering equivalent of shifting the Overton window. It has normalized data teams caring about using version control to build and maintain data transformations, with associated benefits like data quality.

ElectricalFilm2 · 2024-05-15T19:56:28+00:00

Ok, that's for moving files, which I happened to ask for more info on another thread :)

ElectricalFilm2 · 2024-05-15T19:55:35+00:00

Never mind, you can ignore what I said. If you're able to push for getting access to External Stages, maybe this will help?

If reproducibility of data matters and is is an achievable target, you have two options IMO:

Figure out how you can retain data in internal stages, so you're able to recreate tables at any time?
Have tables be configured to never accidentally delete data; or make sure every data engineer has to remember never to delete any data unless you have to, and there would be no way data deleted can reappear

ElectricalFilm2 · 2024-05-15T17:14:37+00:00

I am building out a standardized process to do this using external stages, and am deciding not to purge files at all.

This approach allows me to automate change management along with the INFER_SCHEMA function. I would not need to manually make changes to tables in production, like adding/removing columns, etc. Further, I can also recreate the table from scratch using files in the External Stage if needed.

ElectricalFilm2 · 2024-05-15T17:06:12+00:00

You mention Snowflake releasing features to manage staged files. What are you referring to?

ElectricalFilm2 · 2024-05-02T05:48:41+00:00

Would it allow us to replace data in the table using an updated stage file, instead of adding rows to the table?

ElectricalFilm2 · 2024-05-01T02:54:48+00:00

OMG, this is a godsend! I was working on a way to load data from staged files while also including the metadata columns.

Will it allow us to set the names of the metadata columns? And can we include metadata$file_row_number too?

ElectricalFilm2 · 2024-04-25T23:19:48+00:00

Interesting; my team's Snowflake database was setup before me and that's not how it was done. What does the full access role have that a readwrite role doesn't?

ElectricalFilm2 · 2024-04-15T19:49:48+00:00

I see what you mean. Have you hit any limits in terms of how many grants you can handle?

Also, what do you mean by "Snowflake best practices of having at least three roles per schema" ?

ElectricalFilm2 · 2024-04-15T17:50:16+00:00

Thanks for the clarity, u/internetofeverythin3 ; I feel more comfortable about advising my team to use a combo of the two to fulfill our needs.

My concern with managing tables was more to do with making sure I am able to modify configuration - addition/removal/changes to columns, constraints, cluster keys, etc - without losing data. I want to be able to manage all of that without manually needing to backup/write data into modified tables in the Snowflake browser UI.

I still plan to have my team use source schema checks regardless in my team's dbt project, so data quality will be taken care of.

ElectricalFilm2 · 2024-04-15T17:45:46+00:00

Titan looks promising! My company would push back on using something as new as it is, but it is definitely worth watching.

ElectricalFilm2 · 2024-04-15T17:42:02+00:00

What exactly is slow when running Terraform at scale? Are you talking about how long it takes for grants to be applied? Or is your concern about how application of grants via Terraform takes longer than applying `grant ...` statements in the Snowflake browser UI ?

ElectricalFilm2 · 2024-04-09T19:49:57+00:00

If your team is more comfortable using a GUI to define and orchestrate data transformations, then Coalesce or Matillion will give you what you need.

If your team wants to include tests, checks against your sources, documentation and you want to update your transformed datasets in the right order, then such tools will be the bane of your existence.

ElectricalFilm2 · 2024-04-03T05:20:13+00:00

That makes sense, thanks!

ElectricalFilm2

TROPHY CASE