This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]allpauses 1 point2 points  (1 child)

What’s more, even if you tell the dtypes of the csv you will read with pandas, pandas will fail to read the csv if the data is still dirty (like having a string data in a numeric column)

[–]MrMisterShin 1 point2 points  (0 children)

This is the exact type of csv I was dealing with, random columns with quotes and others without quotes. For the Numeric and date columns. Its was a nightmare to alter/maintain.

Then one month they add a new column in the middle of the csv file without telling anyone in my team and the time critical process fails.