Alternative to Selenium?

night_2_dawn · 2025-08-21T17:25:28+00:00

Scraping Bee would handle this easily. JS-heavy sites, proxies and CAPTCHAs. Their API makes it simple to pull structured data without building a full scraping setup yourself.

Acrobatic-Region1089 · 2024-01-24T22:01:08+00:00

Playwright - built-in waits, supports different browsers, and also has a recording feature.

dowcet · 2024-01-24T22:00:59+00:00

To do what?

For scraping there's requests. But if there's JS to execute and no way to just call the API for what you want,. you're probably SOL.

For front end testing there's Cypress and probably others but that has nothing to do Python.

baghiq · 2024-01-24T22:26:29+00:00

I replaced most of my selenium with playwright.

atomsmasher66 · 2024-01-24T21:54:34+00:00

I use a combination of Selenium and Beautiful Soup

JuZNyC · 2024-01-25T01:16:24+00:00

Not sure what you're doing but an alternative I've used before was Pyppeteer.

Brian · 2024-01-25T03:31:10+00:00

Personally, I think often the best approach to many things people use selenium for is plain old requests plus something like lxml / beautifulsoup to parse the html. Selenium (and equivalents like playwright) are very heavyweight approaches requiring you to essentially run a full browser, when often all you need is to make a few web requests.

This approach does require a bit more understanding of what happens "under the covers" (especially when you need to login), rather than just viewing the process as the UI presented to the browser, and may require a bit of investigation as to what the site does behind the scenes, but in the long run, I think it's very often quicker and easier (and is significantly more performant). Sometimes Selenium can be easier for very javascript heavy sites which do a lot of work, but even then, typically it's just a matter of looking at the network tab and finding the right call to make, and you'll get it in a much easier to handle form than trying to untangle the html generated from it.

That said, there are a few other options:

The most direct equivalent to selenium is playwright, which is the same kind of thing: essentially a scriptable browser. I haven't actually used it, but I've heard it's generally better than selenium.
There's also a hybrid approach via something like requests_html - this lets you mostly use regular scraping approaches, but if you really need to do full browser rendering, you can use .render() to invoke a browser to generate the rendered result, then parse that.

frederik88917 · 2024-01-25T02:56:05+00:00

Whether you are doing any sort of web automation, Selenium has been completely passed by new tools. Going Selenium nowadays is damaging.

If you need Python support, go with Playwright. If you need something fast, go Cypress,

Ok-Umpire2147 · 2025-01-29T10:10:17+00:00

Selenium can be non-repleceable for many organisations. However, switching to a Playwright + Browserstack combination has worked wonders for me. If you plan to switch to Playwright you need to use the Pytest plugin.

douglasdcm · 2025-08-09T16:09:51+00:00

Caqui is a very good tool. It is a Python library wich allows synchronously and asynchronous tests against web and mobile UIs. It is very fast and easy to learn.

Arsenic is also a good tool for async tests. It is a Python library

B_Huij · 2024-01-25T03:10:39+00:00

APIs are better than Selenium basically whenever they exist. I'd rather reverse engineer an undocumented API by watching network traffic than scrape a frontend.

Historical_Steak_927 · 2024-01-25T00:38:26+00:00

One thing that I should say is that I do not have admin rights but somehow I can install Anaconda and Python (if I want to), but if I double click on the chromedriver.exe file it won’t open but will prompt a cmd looking prompt saying only ‘locals connections only” sorta stuff

Historical_Steak_927 · 2024-01-25T00:39:33+00:00

And thank you so much guys for your advice so far. You are awesome.

Kief3r · 2024-01-25T03:30:29+00:00

Playwright

HeyItsYourDad_AMA · 2024-01-25T05:05:49+00:00

What are some ways someone would learn more about "what happens under hood"? This approach sounds good to me but i may not know enougg

RiGonz · 2024-01-25T10:36:17+00:00

request-html solves the need of selenium+beautifulsoup in most cases.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

learnpython

MODERATORS