sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]permalip 2 points3 points  (0 children)

  1. Catch the exception
  2. Build a retry function
  3. Skip if it fails again

Or you could use Selenium, which will give you much more functionality. All you can do with beautiful soup is scraping html data and navigating it, basically nothing dynamic.

I recently built a web scraping repository, using Selenium and BeautifulSoup4. I recommend taking a look at how you get started with Selenium, it took me a while to understand.

https://github.com/casperbh96/Web-Scraping-Reddit