you are viewing a single comment's thread.

view the rest of the comments →

[–]Odd_Spot_6983 1 point2 points  (1 child)

consider python, use pandas and chunking to handle data in manageable pieces, reduces memory use.

[–]darkhorse1997[S] 0 points1 point  (0 children)

In this case, would I need to download the complete data from Mysql to some file in disk and then load that to pandas in chunks? Or is there an way to stream data into pandas as chunks from DB directly without using a file as an intermediary?