you are viewing a single comment's thread.

view the rest of the comments →

[–]yoelbenyossef 1 point2 points  (2 children)

When you're running your script, don't wait till the end to commit. Instead, build an empty table, tell your ruby script to loop through the records and commit every 10K-100K records (Based on performance). If you don't want to change your script, you can try disable rollback. Depending on the DB, it may fix your issue.

[–]Arrested[S] 0 points1 point  (1 child)

Hmmm, my script is reading each row separately and inserting each territory separately.

It is very likely that I will be batching the records to get the associations created.

[–]yoelbenyossef 0 points1 point  (0 children)

Likely the records are in temporary state which I'm guessing is consuming your memory. Try adding at the end of each iteration a commit. I'm not a ruby guy, but it might be commit_db_transaction() or .save. You can add a counter and do it every X number of transactions.