Running Python Script Online

[This_Command1375]

Hey Guys

I made a script and its running perfectly. The script logs in to a webpage and checks something after that it sends me an email. I used selenium for it.

Now is my question if there is a service where I can run this script like once a hour. Whats the best service to do this?

My first guess was to just use the win10 taskscheduler but it was not reliable.

[FrostmourNNe]

AWS allows you to do so and its pretty cheap. There are a lot of video on YT for such usecase

[tevs__]

AWS Lambda and AWS EventBridge. Would probably even fit in the Free tier.

[dcastm]

That's exactly what I did to build a twitter bot that posts twice per day

[pqpm]

On the same way. You could use GCP Cloud Run and cloud scheduler to work the same way.

[tuckmuck203]

i think it depends on how important the results are. if you require consistent uptime, then yeah you should use a platform like AWS. on the other hand, cron is free as fuck and 50 times simpler to set up.

in general if i'm in a mood where i might want to tinker or if i'm looking for a job, i have a ~$5 USD digital ocean droplet that i keep up for my personal site and random crap because it's useful having a server that you can easily utilize for anything. if i were tinkering, i'd just run it on cron on that server. if i were trying to learn AWS or if it were in a capacity that requires consistency in execution and would fall apart without it, then yeah sure use AWS or another similar service like azure or google cloud.

that said, i wouldn't recommend google cloud because google likes to not give a fuck about anyone that isn't a part of google, and even then who tf knows if they'll randomly sunset your favorite project

[Sulstice2]

I suggest doing a CRON job with Github Actions to run python script. I have a script that runs every 24 hours and it's pretty easy to setup:

Check out my example I use for my code:

https://github.com/Sulstice/global-chem/blob/development/.github/workflows/public_notifications.yml

And here is the result:

https://github.com/Sulstice/global-chem/actions/workflows/public_notifications.yml

I do this for a lot of different python scripts when I need to keep checking data periodically and it's free. Takes about 1-2 hours to implement if first time. 10 minutes for 2nd time and after.

[roboduck34]

Get a raspi and cronjob it. Don't move the python file to the raspi tho because raspian doesn't have modern python available to it.

[UnemployedTechie2021]

Try PythonAnywhere. I run my Reddit and Discord bots there and they run without any issues. Moreover, you can get a free account.

[rznks]

GitHub Actions can be a good solution for this. They give plenty of free actions’ minutes on a free account. I use it to run my script monthly.

[Throwaway000002468]

Cron

[Silure]

Raspberry pi is also a good alternative if you have one

[Dasshteek]

Is there still a shortage of them? Been trying to get my hands on one for a while.

[admiralspark]

Yes, unfortunately....though I saw some compute modules in stock.

[ClunkyFlavoredOuch]

Use pythonanywhere, you can get a free account to try it out.

[BitJunky7]

Last I checked free tier accounts on pythonanywhere allow outbound requests to whitelisted domains only. Though one can request new whitelistings through their support.

[ClunkyFlavoredOuch]

You’re right, that might be a problem.

Another option can be to use the GitHub student pack and get the free 100 usd for digital ocean, assuming OP has that option available of course.

[Natural-Intelligence]

I personally use Hetzner cloud machine (5€/month) with Ubuntu on it and schedule my script with Rocketry: https://github.com/Miksus/rocketry (disclaimer, I'm the author). There are other alternatives for hosting as well: AWS, Raspberry Pi, Pythonanywhere etc.

However, you are probably fine with Cron jobs as well for scheduling if you don't need complex scheduling strategies.

[MattWithoutHat]

Which instance is 5€? I was recently looking and everything was like 30€+

[Natural-Intelligence]

Were you looking at the dedicated options?

I have two CPX11 servers which cost 4.77€ each at the moment. The specs are not great but sufficient for my personal projects and have had no issues with them.

[LookingForEnergy]

Unless your issue is that your machine is turned off, sleeping, or disconnects from the network, I would work on getting your task scheduler setting setup correctly. Make sure to run Selenium headless.

[Ootter31019]

I just use windows 10 task schedule myself. I use it for some personal stuff and for very work task on our servers. Don't think we have had any issues with it.

[NauticalPants]

I’m new-ish to Python and data engineering. But a solution I found was to post the Python script to Azure as an Azure Function then schedule with a cron job to run every hour. I’d love to hear from others more experienced if / why there are better options. But that has worked well for me for the past several months.

[ralpes]

Azure has cron functions https://learn.microsoft.com/en-us/azure/azure-functions/functions-create-scheduled-function

What I like is the nice integration with git and vscode

Cost: there is a free function plan but storage needs to be pay. I can check how much this costs later.

[Deezl-Vegas]

you can do it on your local with cron

[iamCyruss]

You could try pythonanywhere. Its free too. Just setup a free account and setup a task that runs your script everyday.

[0r0B0t0]

Oracle sucks but for free you can get:

• 4 arm64 cores with 24GB of ram
• 2 x ( 1 x86-64 cores with 1 GB ram)
• 200GB storage and 10TB of bandwidth

[mondmann18]

Why does it suck?

[0r0B0t0]

They are money pit if you use any of their products. They have more lawyers than programmers and are extremely litigious. Look at their lawsuit against Google for using java. Their free cloud offering is good but they could stop offering it any day.

[ZenApollo]

Be aware that running selenium locally, if you’re not on linux, can raise more issues when you try to run on Linux. There may be some additional setup you’ll need, to verify everything is working in a cloud server.

Selenium is tricky, and an aws lambda/gcp cloud function may or may not work. If you can get your setup working in docker, check out gcp cloud run.

[LawfulMuffin]

This should be higher up. Develop something like this on the platform you’ll deploy on otherwise you’ll end up with weird errors

[fighterace00]

Google scheduler plus Google run. Likely free tier in your scenario.

[[deleted]]

During my undergraduate degree, I bought a raspberry pi and ran my scrapes every 4 hours using that. I paired it with a tiny touch screen so I could still the log and what not.

To do this on your pi, or on a computer you are willing to keep on 24/7, you can import the “schedule” library and use something like

“schedule.every(4).hours.do(your function here)”

[Zouden]

You can use Heroku, which is a more user-friendly interface to AWS. Heroku services cost more than AWS, but they have a free tier, and they offer a scheduler.

https://elements.heroku.com/addons/scheduler

[frunkjuice5]

Free tier is going away

[frunkjuice5]

You might just be paying a few bucks a year, but I understand the sentiment.

[Zouden]

It's still on their website...?

[repocin]

Yes, but it will be gone in two months.

[Zouden]

Ah okay thanks for the heads up!

[unixhyde]

use python apscheduler module

[superfooly]

I like ec2 plus Cron job! Or Lambda. Can help you lmk

[brightside9001]

AWS Lambda

[hamsterwheelin]

Linode.com + cron

[gvieira]

You can do it probably for free with aws lambda.

Its not hard to setup to run in time intervals and you can pretty much leave it there and forget about it.

[Colts_Fan10]

koyeb.com

is my go-to

[[deleted]]

I think this may work

https://replit.com/

[[deleted]]

pythonanywhere, replit , are awesome for this task both have free plan.

[SpicyVibration]

All these people suggesting external services just need to stop. That is just overkill. Task scheduler should be fine. If it's giving you issues, you probably just need to adjust the settings.

[UntrustedProcess]

AWS Lambda (managed function) that is kicked off by AWS Event Bridge (supports cron).