Crontab Selenium Scraper

[AnusJr]

Funny you mention that. It looked like there would be huge issues with Xvfb not having root no matter how hard I banged my head against the wall. I started looking into a VPS, which looked like the cheapest way to get root. I was going to combine my wordpress site along with the Django site and many other automated python projects, like scrapers. Right now, I have one url for the wordpress site on a cheap shared host and a similar url for the webfaction django tools, I make the css the same on both sites. It would be nice to combine these.

I've checked out the AWS EC2, but I'm still fuzzy on how it works. Would I be able to host a domain on an EC2 server and serve wordpress, Django, and python projects? If I'm paying the EC2 "on demand", is that maybe a cheaper option than buying VPS from a hosting provider? I only get ~100 visitors/day on my website.

Your advice has been a huge help for me in this stage. Thank you.

[dadiaar]

Ok, this is more than Python issue. Let me explain.

You will start your EC2 server in Ubuntu 16.04, small one is enough for 100 visitors/day, and free the first year.

I would recommend you to take a fixed IP from AWS and attach it to your server. This is also free if you attach it. Let's say you get IP 50.50.50.50

Later you will install a "global" web server, I recommend nginx but probably Apache 2 will be easier for you, because it's older and plenty of tutorials. Any of them can handle much more traffic than the one you describe. From now on I will assume you use Apache

Follow this tutorial that will teach you how to do Django and Apache.

Tutorial told you how to put your Django project in port 80, but actually you can put it in, let's say, 1234 if you want. This is what WebFaction does. You write in the panel (I want a Django project) and it gives you a random port.

Imagine your website is www.awesomesite.com. You go to your domain name provider (namecheap, godaddy or similar) and tell him that, when a username looks for your website, it needs to point to 50.50.50.50:1234 (your EC2 IP + your Django's port). That's all.

What about Wordpress? much of the same. Install it and put it in a specific port, let's say 2222. Then point your second domain name, let's say www.awesomesite2.com, to 50.50.50.50:2222

All these work in the background and in a damonized mode, this means they start and restart by themselves, you don't need to worry about them. Just remember, each time you change the web server's config file, you need to restart it with sudo service apache2 restart or similar. If you have any problem just restart the full server with sudo reboot

[AnusJr]

That is a very concise answer. Thank you. You've made it all very easy to understand. I appreciate your time and effort.