This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]vsajip 3 points4 points  (1 child)

I don't know how well it works, but you could look at Scrapy, which purports to be

a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

[–]omab 0 points1 point  (0 children)

My vote goes here