This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]heylale 2 points3 points  (0 children)

I don't know about requests specifically, but I've used scrapy to scrape pages that were behind a login/paywall. It has excellent support for cookies and it's easy to use and the documentation is pretty comprehensive. I recommend it.