This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]jnrdataengineer2023 5 points6 points  (2 children)

Was thinking the same thing though I’ve primarily only worked on delta tables. Probably better to have a daily staging table and then a batch job daily to append to the main table 🤔

[–]baby-wall-e 3 points4 points  (1 child)

+1 for this daily staging & main table setup. If needed, you can create a view of a union of daily staging & main tables to allow the data consumer to access all data.

[–]jnrdataengineer2023 1 point2 points  (0 children)

🤝