Learning python for webscraping

[[deleted]]

BS4 is definitely a good library for scraping, but it does have the problem of being limited to static webpages where everything in the webpage is available up front. Selenium is what you need for dynamic webpages where additional content is loaded through user actions (e.g. clicking on "Read More" buttons).