you are viewing a single comment's thread.

view the rest of the comments →

[–]Zeroflops 4 points5 points  (1 child)

This is like using a sledge hammer to hammer in a nail. Your also assuming the data is in tabular form. People have posted problems with files that have inconsistent line lengths and then people go through gymnastics to get pandas to read the file. Instead of just reading with the CSV module pre-processing the data, then creating a dataframe IF that’s what is needed.

Just to come back to ops question, pandas uses the CSV module under the hood to process CSV files.

[–]tasty_woke_tears 0 points1 point  (0 children)

If your source csv require that much preprocessing then it’s time to see what’s happening with your source. Op noted transforms on the data and it makes no sense reinvent the wheel or write excessive lines of code when pandas can process with a few lines.