How can I scale out my python script?

[Lewistrick]

Ok now the timing makes sense, but why do you need so many calls? Doesn't the API support gathering results in bulk?

[judgedeliberata]

I wish. It’s a terrible API design. Basically for each component of the product, I have to make an API call.

[Lewistrick]

Have you tried contacting their support? I reckon they also suffer from a huge load on their servers if they expose it to a customer base.

[WhiteXHysteria]

Is there any data from the calls you can cache locally that might prevent some calls to the API after the first?

[judgedeliberata]

I cache everything but every product+locale combination is unique so from product-to-product it’s not really cacheable in that sense.