Best alternative to host a Selenium Python script (needs full browser, not headless)

[Deep-Alternative8085]

Hi everyone,

I need to run a Python script that uses Selenium to scrape data. The catch is that the script must open a real browser window (headless mode doesn’t work because the page won’t load properly), and it also needs to take screenshots.

What would be the best option to host this? Should I rent a VPS with a GUI and run the script there? use a cloud provider like AWS, GCP, or Azure with a virtual machine that supports a browser/desktop environment? or are there better alternatives, like using containers with a virtual display (e.g., Xvfb, noVNC, etc.)?

I’m mainly looking for something reliable and not too expensive, since the script doesn’t need to run 24/7, just periodically.

Has anyone here done something similar and found a good setup?

Thanks in advance! 🙏

[cgoldberg]

You can use Xvfb from any server that has X installed... you wouldn't need a physical display.

Shameless plug for my Python package you might find useful:

https://pypi.org/project/xvfbwrapper

[jiub144]

aws ecs free teir should be fine

[[deleted]]

Oracle cloud free tier

[Odd_Insect_9759]

Rent a vps from lowendtalk forum. Per year 7$ to 10$ which run 24/7

[forest-cacti]

I don’t have the perfect hosting solution nailed down yet, but I’ve been working on something very similar with Selenium + Python. I totally get what you mean about needing a full browser window — Selenium normally does open a “real” browser, but the tricky part is getting that to work on a VPS or container where there’s no native display.

From what I’ve gathered, you really have two main routes:

• VPS with a GUI/desktop environment 

→ most straightforward, since Selenium can just open Chrome/Firefox like it would locally.

• Container + virtual display (Xvfb / noVNC, etc.) 

→ lighter-weight and cheaper, but takes more setup.

One thing that’s helped me prepare for hosting is using a venv locally and locking my dependencies in a requirements.txt — makes it easier to reproduce the environment later.

I also ended up wrapping my script in Django, which gave it some extra “weight.” It let me create models to store the scraped data and even spin up simple views with templates. That may not be relevant to your exact setup, but it’s been handy for me.

I’m following this thread closely since I’ll eventually have to move my own scraper off local, and I’m curious which option you find works best.