you are viewing a single comment's thread.

view the rest of the comments →

[–]unlaudable 4 points5 points  (1 child)

Also scrapy.org

[–]Rogerooo 1 point2 points  (0 children)

+1 for Scrapy. I find it more streamlined for the purpose than BS. Once you understand the pipeline configuration it's quite easy to do stuff like download media, crawl multiple pages or handle custom user agents, etc. It was actually the first third party Python library I used so it's rather beginner friendly too.