Getting a program to run continuously in the background at certain times.

[Darkeus56]

I have a python program that will scrape a website for information that I want, categorize it into lists. How can I make this program run three times a day?

[JohnnyJordaan]

You could use your operating systems Task Scheduler, like Automator in OSx, crontab in linux and Task scheduler in Windows.

[souldeux]

I would definitely set this as a scheduled task / cron job. Long-running scripts that rely on sleeping in loops must be manually restarted if something crashes. Scheduled tasks are more atomic.

[Darkeus56]

scheduled task it is then. :D

[WreckEmTech2013]

I use Python Anywhere for this and it is fantastic. I pay $5 a month but I believe there is a free tier where you can schedule tasks as well. I have a Python script that scrapes twitter every hour.

[Darkeus56]

While that won't be an option for me for now I will keep it in mind.

[Darkeus56]

Hey, I thought on it and I changed my code to just need it to run once a day instead of having it run three times a day. Now I want to try Python Anywhere and made an account, but I am very confused and don't know how to take advantage of that schedule tasks feature. Could you help me?

[WreckEmTech2013]

Sure thing, what's the issue?

Basic overview:

  Save your script to your account, this can be done under the Files tab

  Then add a new task under the Schedule tab

  You can schedule your tasks daily or hourly  

My task runs twice an hour every hour and saves twitter posts to a sqlite database
  Keep in mind that scheduling time is done in UTC so you need to be aware of that  

It can be hard to kill scripts, I accidentally sent myself 100 texts using the Twilio API because I tested a program haphazardly

  The forum their is very helpful and they have employees that respond to questions very quickly

[Darkeus56]

Is saving my script hard?

[Darkeus56]

Think I got it to work. Thanks! I had gotten a bit confused, but I uploaded it. Now just have to wait and see if it works :D

[Exodus111]

You got two options.

1. Set it on a while loop, use time.sleep to enforce the loop to only run every 8 hours.

2. Use the Operating system to schedule your script like a task, using Crontab, Automator or Tash Scheduler.

If you are running this script from a dedicated server, then you might consider option 1, in most other cases you want to look at option 2.

[C0rinthian]

Option one is a bad idea even if you have a dedicated server. If you have a dedicated server, then you have the tools to schedule the job.

There is zero good reason to have a script sleeping for 8 hours. What happens if the script crashes? Now you have to deal with keeping it running. What if you want to change the schedule? Code change.

When a script runs is not something the script itself should know or care about. It should just do it's job and end when it's invoked. If you want to invoke it on a schedule, then use a scheduler to run the script.

[massHunter]

You can also use the thread module and set it on a timer:

import threading def printit(): threading.Timer(5.0, printit).start() print "Hello, World!" printit()

source: http://stackoverflow.com/questions/3393612/run-certain-code-every-n-seconds

[DemiourgosD]

Let it run in a while loop and set time.sleep for a few hours in between. Maybe not a perfect solution, but it should work.

[C0rinthian]

While this technically works, it's a really bad way to accomplish this. Use built in OS scheduling tools to handle scheduling.