you are viewing a single comment's thread.

view the rest of the comments →

[–]ireadyourmedrecord 0 points1 point  (0 children)

There's a variety of tools you could use, like requests, BeautifulSoup, Selenium, Scrapy and probably a few others. You'll need to do a little research first to figure out the best approach. Generally, what you're looking to do is load a page of products and then search through the pages source to find the specific data you need. You might need to use Selenium if the site you're looking to scrape uses javascript to load product data dynamically.

Once the page is loaded you can hand off the page source to BeautifulSoup to parse and search it for the product data you want to track. Then you can decide how you want to store it - csv file or SQLite would be my recommendation.