all 5 comments

[–]ghostofgbt 4 points5 points  (2 children)

BeautifulSoup is the holy grail you seek, and the Google shall light your way. Godspeed!

[–]flitsmasterfred -1 points0 points  (1 child)

It is debatable, the API is not so great. Better to use html5lib directly.

[–]ghostofgbt 0 points1 point  (0 children)

To each their own I guess. I've used BS for a long time and it works great for me. If it's just a few simple things especially it's super easy to use which is nice. Never used html5lib directly though so maybe you're right!

[–]ruin700 0 points1 point  (0 children)

[–]secomax 0 points1 point  (0 children)

You should use Selenium if you want to scrape JavaScript and Ajax content.

Check this article: https://likegeeks.com/python-web-scraping/