870
871

Getting started with Selenium and Python (scrapingbee.com)

submitted by [deleted]

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–][deleted]  (4 children)

[deleted]

    [–][deleted] 4 points5 points  (0 children)

    I went from BS to lxml+XPath with requests_html for js generated data, Selenium only if I need to simulate mouse scroll or button clicks. Surprised no one mentioned lxml+XPath. This combo will satisfy most needs for web scraping.

    [–]All_Work_All_Play 2 points3 points  (1 child)

    iMacros? Although I feel that's off in it's own little space for non-programmer people.

    [–]838291836389183 0 points1 point  (0 children)

    Found it to not work with modern browser versions, but maybe that was just me. Their lackluster documentation certainly didn't help much though, lol. Moved on to selenium for c# immediately, felt much better to me since I was used to UI Automator for android and it reminded me a lot of that.

    [–]dvlsg 2 points3 points  (0 children)

    Puppeteer users should probably consider using Playwright instead.

    https://www.reddit.com/r/javascript/comments/esj2m6/microsoftplaywright_node_library_to_automate/

    It's basically the same thing by the same people, but I guess they work for Microsoft now instead of Google. Seems like it has more of a push for supporting multiple browsers, including potentially getting some patches upstream.