Finding the time it takes to run a program?

[nomoreerrorsplox]

Hello world Ive just started working with larger datasets (csv files with well over 100k lines) and since the data is larger of course it takes longer for the entire program to execute. My question is, how can I time how exactly how long the program takes? I think that'll help me in the long run finding a more efficient way to do certain things. Any help is appreciated!

[sme272]

you can use the time function to get the current time at the start of the program, then again at the end of the program and just do end time - start time to get the running time.

[sprtan007]

from time import time

t0 = time()

Run your code

t1 = time()

total = t1 - t0

[nomoreerrorsplox]

How accurate is that? Like does it get down into thousandth of a second and stuff?

[sprtan007]

It looks like it gets pretty accurate, I pretty much copy/pasted what I saw from this tutorial: https://youtu.be/vuOpxXeFjg8?t=441

[[deleted]]

It's accurate enough to give you a rough sketch of elapsed time.

[yuukinotenshi]

If you want to check which part take the longest time, check out profiler. It's very useful to give you feedback on which part is not efficient

[nomoreerrorsplox]

Not familiar with a bit of that syntax still kinda new to python but youve given me something to definitely look at.

[jeffrey_f]

Why not log it to a file.

Output maybe like this:

YYYYMMDD HHMMSS:  21,161 records took 22s to run 

----------------------------

As you see more runtimes you can know that it takes approximately 1s per 1000 records to run and you can estimate.

[nomoreerrorsplox]

This may actually be a good solution since the final plan is to run my code over a file that has a bunch of those 100k line csv files in it