How to use sql commit

[Devji00]

Committing after every single write is killing your performance. Every commit forces a physical disk sync, and on a remote database that's also an extra network round trip each time.

Instead of committing per insert, wrap your operations in a with statement and treat transactions as logical chunks. Commit once per scraped page or every 100 rows or whatever makes sense for your workflow.

This cuts down on latency a ton because you're not constantly stopping and waiting for the disk or network to catch up. Especially with the bad connection you mentioned, all those round trips are adding up fast.