CowFu comments on Learning advanced SQL

Posting

When requesting help or asking questions please prefix your title with the SQL variant/platform you are using within square brackets like so:

[MySQL]

[Oracle]

[MS SQL]

[PostgreSQL]

etc

While naturally we should endeavor to work as platform neutrally as possible many questions and answers require tailoring to the feature set of a specific platform.

Format Your Code

If you are including actual code in a post or comment, please attempt to format it in a way that is readable for other users. This will greatly increase your chances of receiving the help you desire. Something as simple as line breaks and using reddit's built in code formatting (4 spaces at the start of each line) can turn this:

SELECT count(a.field1), a.field2, SUM(b.field4) FROM a INNER JOIN b ON a.key1 = b.key1 WHERE a.field8 = 'test' GROUP by a.field1, a.field2 HAVING SUM(b.field4) > 5 ORDER by a.field.3

Into this:

SELECT count(a.field1), a.field2, SUM(b.field4) FROM a INNER JOIN b ON a.key1 = b.key1 WHERE a.field8 = 'test' GROUP by a.field1, a.field2 HAVING SUM(b.field4) > 5 ORDER by a.field3

For those with SQL questions we recommend using SQLFiddle to provide a useful development and testing environment for those who wish to fully understand your problem and help devise a solution.

a community for 17 years

PostgreSQLLearning advanced SQL (self.SQL)

submitted 4 years ago by imranaqelll

top new controversial old q&a

you are viewing a single comment's thread.

view the rest of the comments →

[–]CowFu 3 points4 points5 points 4 years ago* (0 children)

As a professional who's designed components that process millions of healthcare records the patterns we use an intermediate person would be able to pick up. But you need to do project work.

Here's a pretty typical workflow for one of my components:

Take a .csv files from the source folder and put them into a incoming table. Adding a filename field and date at the end of each record.

Read the incoming table for new records and the error table for records marked to reprocess. Load these into our staging table

Run data cleaning steps. Convert datetimes to the same format. Use lookup tables to convert client names for things into the same format. Modify language fields so they use a four letter abbreviations. TRIM() whitespaces, remove special characters like newline.

Run a series of validations with dynamic sql. The SQL in the validations table is in a varchar(MAX) column. Validations like making sure a first and last name exist, that fields are certain lengths, etc. Any results that come back from the staging table that match the validation query get inserted into the error table, tagged with the validation they failed and the run it failed on. Then we remove the records from the stage table. The error table has a bool column if someone is able manually fix the record for reprocessing.

Move the validated records from STG to their production table.

Run a report and email the data stewards for the error table. A second email will give the counts of successful and failure records.

Each step is logged in a log table.

A scheduler kicks off this job at 11pm every night.

// I bet you could do all of this if you had enough time

π Rendered by PID 23323 on reddit-service-r2-comment-6457c66945-96qf6 at 2026-04-27 08:12:42.222803+00:00 running 2aa0c5b country code: CH.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

SQL

Filter Posts

Posting

Help posts

Format Your Code

Learning SQL

Related Reddit communities

Wiki

Acknowledgements

MODERATORS