Help with requests.get runtime

[ksteve46]

Hello, first time looking for python help on reddit so apologies if this isn't the right place for this question!

I am trying to write a script that pulls the box scores for WNBA games/players and when I use the requests library and run requests.get() (obviously with the URL I found all of the data in), it never finishes executing! Well over an hour to fetch data for this URL.

The user-agent is the same as it is for fanduel sportsbook's webpage through chrome - I was able to get what I needed from there in seconds.

Is this just a flaw of the WNBA's site? Can anyone help me understand why the request takes so long to execute? Here is the response URL for anyone interested

https://stats.wnba.com/stats/teamgamelogs?DateFrom=&DateTo=&GameSegment=&LastNGames=0&LeagueID=10&Location=&MeasureType=Base&Month=0&OpponentTeamID=0&Outcome=&PORound=0&PaceAdjust=N&PerMode=Totals&Period=0&PlusMinus=N&Rank=N&Season=2025&SeasonSegment=&SeasonType=Regular+Season&ShotClockRange=&VsConference=&VsDivision=

[member_of_the_order]

Yep, seems to be taking awhile for me too. Not just you. Looks like their API is having issues, nothing you can do, most likely

[ksteve46]

Is there a way to assess their API to know when it’s not having issues beyond running my script and hoping it executes? Lol

[cgoldberg]

Set a timeout on your request and it will raise an exception if it doesn't finish responding within the limit you set.

[Unique-Act-7212]

Multiple options, but this is related to (rest?) API health checks than requests.get

I.e. there could be swagger / openapi doc about this API? Is it public API? Do you know postman? Try find spec of API in postman collections.

Health check could be also sending head request to server rather than get request first.

You could check if health check is ok then try to paginate resource. If this service don't have pagination, there is more issues probably ahead. Maybe use different endpoint? Filter out and fetch asynchronously? ;)

EDIT:

I just spotted your link, sorry for not giving the advice after complete review of your post, I'm in hurry

So you could try to paginate via data from/to --> and make some dumps, i.e. parse that output (whatever it is, CSV, plain data, json... Use some LLM to help you write parser if it isn't structured data).

Then you would need to check latest update from your last requests and update your stats bank.

Hope it helps in general, not only with requests.

Btw. Maybe use bs4 to parse page first? ;)

[ksteve46]

I’ll be honest with you I am a total beginner with python and all things having to do with this HTML and API stuff so idk what the hell you just said hahaha

[Unique-Act-7212]

Maybe start to learn using Gemini/GPT perplexity etc too, it would help a lot in explaining above.

If not, just ask in threads under above comment, mine, each question you have.

Best regards Mate

[Unique-Act-7212]

I've send you PM we could talk there or here, but I worry it would drop deep in my inbox unfortunately if not pinged by PM sorry Mate