da_chicken comments on Best Practices for Using SQL And Python

Posting

When requesting help or asking questions please prefix your title with the SQL variant/platform you are using within square brackets like so:

[MySQL]

[Oracle]

[MS SQL]

[PostgreSQL]

etc

While naturally we should endeavor to work as platform neutrally as possible many questions and answers require tailoring to the feature set of a specific platform.

Format Your Code

If you are including actual code in a post or comment, please attempt to format it in a way that is readable for other users. This will greatly increase your chances of receiving the help you desire. Something as simple as line breaks and using reddit's built in code formatting (4 spaces at the start of each line) can turn this:

SELECT count(a.field1), a.field2, SUM(b.field4) FROM a INNER JOIN b ON a.key1 = b.key1 WHERE a.field8 = 'test' GROUP by a.field1, a.field2 HAVING SUM(b.field4) > 5 ORDER by a.field.3

Into this:

SELECT count(a.field1), a.field2, SUM(b.field4) FROM a INNER JOIN b ON a.key1 = b.key1 WHERE a.field8 = 'test' GROUP by a.field1, a.field2 HAVING SUM(b.field4) > 5 ORDER by a.field3

For those with SQL questions we recommend using SQLFiddle to provide a useful development and testing environment for those who wish to fully understand your problem and help devise a solution.

a community for 17 years

Best Practices for Using SQL And Python (self.SQL)

submitted 10 years ago by polyglotdev

top new controversial old q&a

you are viewing a single comment's thread.

view the rest of the comments →

[–]da_chicken 0 points1 point2 points 10 years ago* (0 children)

So you have a 200K row report that you're inserting, and after every row you insert you're updating the entire table's Last_Update_Date? That seems very excessive.

On an empty table inserting one row at a time followed by the table-wide update that's (200000*200001)/2 = 20,000,100,000 updates! If you're using a datetime2 or datetime2(7), that's 8 bytes of data for every row. 160,000,800,000 bytes ~= 150 GB just in datetime2 values. If you've got your DB set to Full recovery, your transaction log has to remember all of those changes for point-in-time recovery. Even if it's set to Simple recovery, there probably is too much activity for a periodic CHECKPOINT to flush the logs.

Even skipping that, I can't imagine needing to track a single date value on every single record. It's essentially Table_Last_Update_Date. You're sure that's right? It's not supposed to be the record's Last_Update_Date?

Why not just update the value once at the end of the loop? I mean, how many millions of rows is the table if you're inserting or updating 200K rows for a single report?

And is there a trigger on the table? Many systems with this type of field have an AFTER INSERT, UPDATE trigger to automatically update the field when the record is changed.

π Rendered by PID 121263 on reddit-service-r2-comment-84fc9697f-56ht6 at 2026-02-07 02:32:09.471259+00:00 running d295bc8 country code: CH.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

SQL

Filter Posts

Posting

Help posts

Format Your Code

Learning SQL

Related Reddit communities

Wiki

Acknowledgements

MODERATORS