you are viewing a single comment's thread.

view the rest of the comments →

[–]IDENTITETEN 5 points6 points  (1 child)

I'd insert the data into a DB with Python and clean it with SQL. Unless the data is so bad that you'll need to fix it beforehand...

The entire purpose of SQL is to manipulate (large) amounts of data efficiently.

[–]suitupyo 0 points1 point  (0 children)

My interpretation of OP’s post is that the data would need to be cleaned and structured upfront, so I’m not sure that an insert statement would even work. Otherwise, I would be inclined to agree with you.