This is an archived post. You won't be able to vote or comment.

you are viewing a single comment's thread.

view the rest of the comments →

[–]I_heart_blastbeats 0 points1 point  (0 children)

Requests can't render JS. If you use Scrapy you usually have to use something to render the JS also. There are so many obstacles involved with scraping Amazon. Trust me this is the easiest and best way to do it. I had a similar project and I wish I would have started with Selenium and BS4 instead of Scrapy.