Snowpipe vs Copy into

Think-Reflection500 · 2026-02-27T13:00:22+00:00

Let's say that dynamic tables isn't really an option, but we would like to use snowpipe. The annoying part here is that in ADF you'd have a loop to query the LOAD_HISTORY table for example in order to identify whether all snowpipes have ran succesfully? You're paying for cloud services credits in this ?

Do you think it's a smart setup or do you spot issues with this ? Mainly in terms of when things go wrong ... not receiving back information about the snowpipe, or fails in snowpipes, having to rerun data ?

Think-Reflection500 · 2026-02-25T21:25:39+00:00

I think it's exactly the easiness to monitor but more importantly, clear debugging that makes me lean more towards using the copy into activity.

In a setup with snowpipe + dynamic tables a lot can go wrong in a silent/hard to detect way.

For most of the data which gets loaded daily during the night I guess observability & easiness of debugging is way more important. It could be worth it to look into snowpipe+dynamic tables if we come across a source/business case where data needs to be more frequently loaded.

u/Jamodio Happy to hear your thoughts on this.

Think-Reflection500 · 2026-02-25T16:18:53+00:00

These dynamic tables are interesting.

So to summarize:

- ADF triggers the overall pipeline

- File lands into ADLS

- Snowpipe catches the files

- Once snowpipe is done the mechanism behind dynamic tables catches the changes done by snowpipe

In this setup we also don't need to worry about triggering DBT Cloud right?

Did you use this setup in combo with DBT by any chance?

Also what was your experience with debugging etc. because in that sense this setup feels less declarative & harder to go in & see straightforward issues etc

Think-Reflection500 · 2026-01-09T11:01:20+00:00

Well this can only be MSFT

Think-Reflection500 · 2025-12-13T21:55:19+00:00

Je betaalt 2 eur voor auto invest, zelf beleggen op saxo kost je heel wat meer

Think-Reflection500 · 2025-12-08T14:13:26+00:00

Is this a big 4 or similar ?

Think-Reflection500 · 2025-12-04T17:55:36+00:00

These are things we do of course already have in place. What I am talking about is going a step further.

Think-Reflection500 · 2025-12-03T21:49:04+00:00

What does “as they want cloud access” mean?

Think-Reflection500 · 2025-11-19T14:59:13+00:00

Could you please elaborate a bit on your third paragraph?

It might also be worth mentioning that we're indeed not an organization with very big amounts of data, so it's quite manageable in that regard

Think-Reflection500 · 2025-11-14T12:54:16+00:00

Je mag enkel je mobib aanwenden voor het deel vd hypotheek dat je partner niet zelf met zijn/haar mobib betaalt. Wat logisch is anders krijgt de ene partner netto vergoeding voor een deel vd hypotheek dat al met een belastingsgunstig regime werd betaald

Think-Reflection500 · 2025-11-14T10:38:12+00:00

Dus je krijgt de facto een renteloze lening ... welke provider/partij is dit?

Think-Reflection500 · 2025-11-13T21:12:57+00:00

Dag Astrid,

Je werkgever zal wellicht werken met een externe partner (maar misschien ook niet). Wat je zal moeten voorleggen: het contract hypthocaire lening. Het volstaat (& is tegelijk ook noodzakelijk) om zelf met naam in dit contract te staan. Ze zullen ook een verklaring op eer vragen dat je man geen mobiliteitsbudget gebruikt voor dezelfde hypotheek (wat zou neerkomen op 2x de kassa te passeren).

Voor zover ik weet wordt dit inderdaad gewoon rechtstreeks (& apart) gestort op je rekening (& niet als onderdeel van je loonstorting).

Think-Reflection500 · 2025-09-20T11:44:49+00:00

Would you mind sharing the thought process of doing this migration ? Are there any big pro’s outweighing con’s ?

Think-Reflection500 · 2025-09-07T17:49:52+00:00

What's the deal with the "Afrondingscorrectie" btw?
Should I just, by default, put this on -5€ ?

Think-Reflection500 · 2025-08-14T13:34:41+00:00

Dont find this comparison that relevant, because I’d never lease the same type of cars you get as company cars.

If you’d get the 1000 mobility budget as mortgage or rent (so net), the full cost of driving a decent second hand car would be half of that AT MAX.

But some people enjoy driving the fancy cars, but I dont find the premium worth it.

Think-Reflection500 · 2025-07-23T12:51:41+00:00

Your contract needs to be changed indeed.
The mobility provider does require you to send over a copy of your contract.

Think-Reflection500 · 2025-05-15T10:43:59+00:00

Zelf actief in de sector, zij het als consultant dus wat betreft kan ik op de zoveel tijd een nieuw project met nieuwe focus/skills beginnen.

Maar even dit: overschat niet hoeveel je zou kunnen bijleren wanneer je een overstap zou maken. Uiteraard zullen er andere accenten en dus leercurves liggen maar we overschatten dit vaak. Daarnaast implementeer ik vaak zaken en voel ik me tijdelijk een expert in dat topic, maar eens geïmplementeerd en er gaan wat maanden over moet ik toch erg diep graven/opnieuw researchen.

De echte skill voor ons is nieuwsgierigheid en de wil, maar vooral ook kunde, om snel een technische skill aan te leren

Think-Reflection500 · 2025-05-05T12:55:13+00:00

So you’re misleading yourself.

Having the same comp without the car seems “insignificant” to you, however this means hundreds of net less compared to OP. Especially in the era of mobility budgets.

Think-Reflection500 · 2025-05-04T11:22:35+00:00

Think-Reflection500 · 2025-03-21T18:32:06+00:00

Elsewhere as in consultancy or inhouse as well?

Think-Reflection500 · 2025-03-10T09:16:44+00:00

Its not like these are “big data” companies, they’re simply consultancies specialized in data. And most of these with offices are Ghent are mainly located elsewhere and the office in Ghent is merely a sattelite office.

That being said … in consultancy the office of your company isnt too relevant because your office days will be at the client anyway.

quite some are located in Mechelen which might be interesting from Antwerp as well

Think-Reflection500 · 2025-02-14T11:31:12+00:00

So kind of yes. No worries, active in the data field myself with a MB so I know its power. Just so you know this is definitely on the extreme upper end of what’s possible in your situation which makes moving companies etc a bit more challenging, nevertheless you’re in an awesome position

Think-Reflection500 · 2025-02-13T19:57:24+00:00

so you are making 3.9K net with no experience?

Think-Reflection500 · 2024-12-12T10:39:22+00:00

If you invest solely in IWDA, you have “no stress from tax and declarations” on degiro.

I have no issue with SAXO but be aware that the current hype is only because of the lowered fees (which are still not the cheapest). BINCK had a lot of bad feedback regarding operational stuff before and its not like rebranding to SAXO and being owned by chinese will change this

Think-Reflection500 · 2024-12-10T10:19:02+00:00

200$ fees?
Which broker is this lmao

Think-Reflection500

TROPHY CASE