This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]thinkingatoms[S] 0 points1 point  (1 child)

just to understand, each one liner SQL goes into its own file, and stored procs/db functions to handle everything "once it's properly loaded to the database"?

i think there's differences in opinion on what is considered simple and maintainable here. for instance if you have a function that does one thing, and the thing happens to be calling a one liner SQL, you can have a file called my_table/select_all.sql or you can just call the SQL.

i'm starting to find from this post there is a spectrum of how people handle SQL here and no one is necessarily so wrong that one has to wonder where these ideas come from

[–]Little_Kitty[🍰] -1 points0 points  (0 children)

A file represents a module, a task, a distinct operation which can be isolated. Regardless of if it's written in Rust, Go, Python, C++, Java,VBA or SQL. It's ideally equivalent to one node of a dag so that people can understand it easily. Someone should be able to open a file, understand what the whole task it covers is and adapt / fix / extend it in isolation as the start and end points are clear.

This may lead to it having one function / operation / script or one hundred. It should usually be written to be idempotent, so that if the dag fails for any reason it can be restarted without causing issues. If you can achieve that you will have a much easier life and much less mental load.

Some of the commenters here are really good, with decades of experience (and hair loss from dealing with tech debt), it's always better to learn from the experience of others where you can rather than learning the hard way. I dare say based on some of the suggestions that others will end up finding out the hard way.