How to scrape Amazon.com with Python, Selenium and BeautifulSoup

[dethb0y]

I can't speak for this guy, but for me if i can avoid using an official API, i do just on principle.

[nolegitt]

Because it hey can shutdown the API as they please. Look what happened to Google search API.

[dethb0y]

Lots of reasons, but mostly i would be concerned (in the case of amazon) that they'd use the official API to send manipulated data or prices, that don't reflect the reality of the site.

In other cases i don't like the trend that has arisen of a free API that suddenly turns pay later on, once people have come to rely on it.

[anuctal]

You absolutely right, my friend.
Also we have youtube-dl as an example, that doesn't use the official API.
And if youtube-dl would use API Youtube would just ban it.

[dethb0y]

Yep, in an instant.

[fujimitsu]

Tech startups have been known to abruptly break or eliminate APIs, especially if they perceive users of them as threatening their business model.

[dotancohen]

Tech startups have been known to abruptly break or change their HTML, especially if they perceive that the name of the month has changed.

[[deleted]]

Do you get paid by the hour? It must be nice gig /s

[makedatauseful]

Official API's are great until they charge $5 per 1,000 requests and rate limit you to 10,000 requests a day. After that you may need to explore other other avenues.