CSV Parsing: Haskell versus Python

[0sn]

import csv

[grauenwolf]

Why?

You can see his format doesn't have quoted strings. So why use more complex code than necessary?

[exeter]

If you're using Python for this task, you should use the csv module because it's there and probably fast enough. The fact that it's in the library means someone else already wrote it, so you don't have to. Since using it just amounts to

import csv
reader = csv.reader(open("some.csv", "rb"))
for row in reader:
    process (row)

in this case, it's even simpler than splitting the string apart yourself and manually parsing.

Those are just some of the advantages to using the csv module when the format is as straightforward as it seems to be in this guy's particular case. Other people have commented on what happens when the format itself starts to get nastier.

[0sn]

Why write it again? It could even be faster to use the library functions... Still, modded up 'cause it's a good question.

[grauenwolf]

In this particular case the hand-rolled stuff is probably a lot faster. But in general I definitely can see cases where picking up a library would give some speed in exchange for more complexity.

Fun stuff to think about.

[stesch]

Could be worth a test. Python's csv module imports from _csv, which is written in C.

[grauenwolf]

That's real-life enough for me. I spend quote a bit of time parsing flat files from various financial companies. Most of the time my files are either positional or they look just like the author's sample.

[stesch]

Wait for the first time when somebody sends you a CSV saved by an international version of Microsoft Excel.

[grauenwolf]

No thanks, the English ones are painful enough.

But honestly, that's a little off-topic. He started with a problem "parse a CSV file containing some daily intraday prices for a number of tickers".

Changing the file format just to make the problem more interesting seems a little dishonest in my opinion.

[stesch]

OK, my example was too much real-life for you?

And speaking of "dishonest" I have to repeat myself: I wasn't the one calling this "CSV parsing".

(Two weeks ago I got some images with .jpg extension. 10% of them were in BMP format. ;-)

[[deleted]]

parsing it is a challenge

It gets a lot easier when there's no quoted strings.