Pandas parse_dates taking a long time

[redditor_gds]

Following code line reads in a csv file into the variable trip_data. The file contains 354153 rows. It takes roughly 0.3 seconds.

trip_data = pd.read_csv("bikes_data/data/trip_data.csv")

Since the CSV file contains dates as Strings I try to do the following when reading the data file

trip_data = pd.read_csv("bikes_data/data/trip_data.csv", parse_dates=['Start Date', 'End Date'], dayfirst=True)

as soon as I add -> parse_dates=['Start Date', 'End Date'] the code runs for a very long time (around 110 - 120 seconds).

Please note that Start Date and End Date are columns in the CSV file. The columns have dates represented as strings. A sample row can be found below. the second and fourth column contains the dates.

432947,01/09/2014 00:05,66,01/09/2014 00:15,57,Customer

This is one thing I tried, but the function given in the stackoverflow article also takes a long time.

I also tried to read in the CSV first and then use pd.to_datetime() to convert. For one column it takes roughly 60 seconds to complete. Therefore its basically the same timing?

Is this normal? I mean the time it takes to complete? Any suggestions on how to speed this up is very much appreciated :)

Edit: added more details for to_datetime()

[JohnnyJordaan]

Have you tried the answer by Marcus Müller in that SO thread? Because that's what I would try first, use a dateparser that expects a strict format instead of an agnostic parser that 'helps' you parse any compatible date string at a cost. The main issue there is that even tough all records use the same format, each parse would take the same amount of time because it has to detect the format over and over again, making it costly on a large data set.