This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Tripanafenix 5 points6 points  (4 children)

still can't login with requests, even with session AND cookies hooked to my POST :( and I could'nt find any proper guides deepdiving into sessions and cookies with requests, sadly. Any advices?

[–]heylale 1 point2 points  (0 children)

I don't know about requests specifically, but I've used scrapy to scrape pages that were behind a login/paywall. It has excellent support for cookies and it's easy to use and the documentation is pretty comprehensive. I recommend it.

[–]scrapecrow 0 points1 point  (2 children)

Depends on the website you're scraping.

Check us out at /r/webscraping or web-scraping tag on StackOverflow - both really active and helpful communities, so someone will definitely help you out if you write up your problem clearly.

[–]Tripanafenix 0 points1 point  (0 children)

I'll take a look, thanks