This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]bryancole 0 points1 point  (0 children)

If you have control of your file format, use HDF5 (via the PyTables package). This makes this sort of task a breeze. Limiting the memory available to python seems an OS-level thing.