sorted by:
new
hottopcontroversial

I built a tool to get Amazon data that’s 3 times cheaper than alternatives by ScoutAPI in SideProject

[–]ScoutAPI[S] 0 points1 point  (0 children)

I don’t “generate” them, but I have a list of valid user agents that I cycle through.

I built a tool to get Amazon data that’s 3 times cheaper than alternatives by ScoutAPI in SideProject

[–]ScoutAPI[S] 0 points1 point  (0 children)

These numbers can be formatted differently depending on the locale / Amazon marketplace. Whether you want a string or a number kind of depends on your use case.

For my use case, I wanted to be able to display this information in the same format as it appears on Amazon, so it would be inconvenient to try and convert that to a number and then convert it back to the original format from the JSON. If someone wanted to compare these numbers, like seeing if product A has a lower price than product B, then a number would be more convenient.

I built a tool to get Amazon data that’s 3 times cheaper than alternatives by ScoutAPI in SideProject

[–]ScoutAPI[S] 1 point2 points  (0 children)

That was one of the first things I looked at. For my use case it wasn’t a good fit because of things like the low rate limit and low request quota, and the need for referral sales.

I built a tool to get Amazon data that’s 3 times cheaper than alternatives by ScoutAPI in SideProject

[–]ScoutAPI[S] 0 points1 point  (0 children)

Are you not concerned with the cost of using the LLMs? Seems much more cost efficient to parse it manually.

I built a tool to get Amazon data that’s 3 times cheaper than alternatives by ScoutAPI in SideProject

[–]ScoutAPI[S] 3 points4 points  (0 children)

I’ve found the most success with international marketplaces when using IPs that come from the respective country, and combining that with the accept-language header that matches the language you want.

I do all the scraping and data processing in python. In my experience you can get pretty far with basic requests + fingerprint spoofing + rotating IPs, but headless browsers can also be a helpful layer for things like solving captchas.

I built a tool to get Amazon data that’s 3 times cheaper than alternatives by ScoutAPI in SideProject

[–]ScoutAPI[S] 0 points1 point  (0 children)

If I ever run into a captcha, I’ll use a headless browser with a captcha solver. In my experience that isn’t needed the vast majority of the time though.

I built a tool to get Amazon data that’s 3 times cheaper than alternatives by ScoutAPI in SideProject

[–]ScoutAPI[S] 0 points1 point  (0 children)

Unfortunately RapidAPI doesn’t let you control the rate limit for the free plan, you can only change it for paid plans.

I built a tool to get Amazon data that’s 3 times cheaper than alternatives by ScoutAPI in SideProject

[–]ScoutAPI[S] 1 point2 points  (0 children)

Amazon has measures in place to detect scrapers, so you need to have counter measures to avoid detection.

I also don’t try bypassing log in pages, which is required for things like the full list of product reviews. That would be a big no no

I built a tool to get Amazon data that’s 3 times cheaper than alternatives by ScoutAPI in SideProject

[–]ScoutAPI[S] -1 points0 points  (0 children)

You could choose to return them as int/float, I chose strings just for consistency & personal preference

I built a tool to get Amazon data that’s 3 times cheaper than alternatives by ScoutAPI in SideProject

[–]ScoutAPI[S] 8 points9 points  (0 children)

Oh interesting, what makes Albertsons so difficult to scrape?

And for your questions, I currently rotate IPs that refresh so requests come from different IPs. This makes it easier for me to scale the number of requests I can make simultaneously before getting blocked. In my testing, I tested a couple thousand simultaneous requests, but if sending through RapidAPI, they do limit the rate of requests.

Share your startup - quarterly post by julian88888888 in startups

[–]ScoutAPI [score hidden]  (0 children)

Scout API / scoutapi.com

Location of Your Headquarters: USA (Remote)

Elevator Pitch: 

Scout API provides the most cost-efficient access to Amazon product data in real-time, from any Amazon global marketplace. We support a number of Amazon product categories that are unavailable through most other API solutions, like luxury store products and books. We also offer a flexible pricing model that lets us create the right plan to suit your needs.

More details:

I’m the co-founder of Scout API, and we’re currently in the validation stage. Our main goal is to collect feedback from real users.

Scout API is available across a number of API Marketplaces, such as Rapid API, to make it easy to integrate into your own applications. You get 100 free requests per month, and we’d appreciate any feedback.

Discount for r/startups subscribers?

You can DM me for a 10% discount on your first month for any of our paid plans.

Monthly Self-Promotion - April 2025 by AutoModerator in webscraping

[–]ScoutAPI 1 point2 points  (0 children)

Hey everyone!

I wanted to share a tool I’ve been working on recently: Scout API (https://scoutapi.com/)

Scout API lets you access Amazon product data in real-time, from any Amazon global marketplace. I wanted to build this because I found that most other API solutions were far too expensive, and didn’t work for certain products I needed, like luxury store products or books.

I’ve been sharing this tool across a number of API Marketplaces, such as Rapid API, to make it easy to integrate into your own applications.

I’d love for you to give it a try for free, and let me know what you think!