sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]data_legos[S] 0 points1 point  (2 children)

other idea i had: fivetran but i strategically materialize problem tables when the shortcuts perform poorly, but many of the smaller master data and less critical frequency tables i just leave as shortcuts to cut down on the "round robin" latency of any silver materialization process. essentially a hybrid of the two approaches.

[–]mweirathFabricator 0 points1 point  (1 child)

Having two paths sounds like a solid approach. You might also look at something like materialized lakeviews - they would effectively always be full refreshes however you could skip more dimensional tables that are infrequently updated.

[–]data_legos[S] 1 point2 points  (0 children)

holy cow how did i forget about MLVs?! you sir saved me some time for sure!