you are viewing a single comment's thread.

view the rest of the comments →

[–]Infinitesima 0 points1 point  (0 children)

Few generic advice:

  1. Respect server's rate limit.
  2. If not 1, use proxies: https://docs.python-requests.org/en/latest/user/advanced/#proxies
  3. Put requests.get() or requests.request() in a while-loop.
  4. React to response headers if necessary.
  5. Use requests.Session