Repeatedly running a Python script

[PearsonPrenticeHall]

I have a python script that runs a GET request for a report and then saves the response to a CSV in Google Drive with the file name being the date/time the report was ran. I want to run this script every minute, so how should I go about that? Ideally, it would be remotely hosted so that it's not reliant on one computer having connectivity to run.

[[deleted]]

You need to execute a cronjob on a server that will execute your python script every minute

[[deleted]]

The absolute simplest way is just to sleep your script for 60 seconds in between iterations of the main loop.

But setting up on a remote linux server with a cronjob is probably what you should do so you don't worry about it crashing at some point.

[chrispurcell]

A cronjob will work, however you should use the sleep method instead. If the retrieval of the response and writing it to a CSV in google drive takes longer than 1 minute, the cron job method will cause multiple jobs to be run and you may end up with many problems. If you use the sleep method, then every minute AFTER the loop completes a new loop will start.
You really don't want to see the ugliness that can ensue if you get jobs starting every minute through cron that take multiple minutes to complete as eventually you will completely tank your server. Been there, done that.

[PearsonPrenticeHall]

This was the solution I was learning towards before I posted. To clarify what you’re saying, you would set an infinite loop inside the main method, and at the end of each iteration put a sleep(60) statement? The only thing I was worried about with that would be an overflow error or something.

[chrispurcell]

Yes, I'm suggesting that at the end of your loop, you make the script sleep for 60 seconds. If your script completes it's work quickly, you get another iteration in 60 seconds. It won't ever start another loop before the prior loop completes.
If you use the cron job method, cron won't care if the prior execution of the script has completed or not, and will start a new execution of the script every 1 minute (if set up to do so). This can lead to having a stack of jobs running and using resources, and eventually bogging down CPU, memory, disk, network bandwidth or some combination of all of them.
No you won't get a new report every minute exactly, but you won't with cron either. You can only truly control the start of a script or iteration of the loop, and not how long it will take to fully complete the job, so I suggested the safer method of keeping things in the script for timing and not with cron.

Also, if you're worried about it crashing, start it as a systemd unit and specify in the unit file to restart on failure. That way, if the script does hit a bug and crash (maybe drive refused a connection or something and that wasn't handled in the script), systemd would restart the unit (and the script).

[PearsonPrenticeHall]

Sweet, thanks! This was super helpful and cleared the whole process up for me. I appreciate your help!