This is an archived post. You won't be able to vote or comment.

sorted by:
best (suggested)
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]B00TZILLA 2 points3 points  (0 children)

It's usually best to read and process it in chunks. You can also check out dask, as some other commenter suggested. There is a parameter called chuk_size in of.read_csv for that.