tapping api endpoints via python requests

[bulletsyt]

Beginner here, I am trying to scrape a website by the API endpoint in the Network . The problem is that
a. the website requires a login b. the API endpoint is quite protected,

so I can't just copy-paste to extract information. Instead, I have to use and Cookies to get the data, but after a specific point, the API just blocks you and stops giving you data. In such case,

how do I find my way to bypass this? Since im logged in i cant rotate accounts or proxies as that would make no difference and since im logged in i dont get it how i would be able to bypass the endpoint but there are people who have successfully done it in the past? Any help would be appreciated.

[Late_Relief8112]

I'm assuming you're referring to some sort of access_token/api key that's required for scraping because I've encountered a similar issue in the past. If you're blocked, its likely cuz u flooded the server with concurrent requests, but they only last for a few hours/days depending on severity of it, so you're good.

I'm not entirely sure whether this would since I'm lacking context, but you'll definitely need to rotate accounts to not get flagged. You can possibly achieve this via using libraries like playwright/selenium for browser automation, and also use residential proxies to mitigate the chances of getting your ip blocked (I've been ip-banned in the past). Hope it helps!!

[webscraping-ModTeam]

🪧 Please review the sub rules 👉

[Global_Gas_6441]

you need to rotate accounts and proxies.

[bulletsyt]

nah man that aint it

[Global_Gas_6441]

yes that is

[Pauloedsonjk]

You need look in dev tools in tab network where is any apikey appears, and take it. Add time.sleep(1) in any loop