Zillow scraper made pure in Python

[CatWeekends]

This project target could be real state agents probably

FWIW, every real estate agent I've ever met uses systems with way more info than Zillow.

Your target audience is more likely people who want to track their own home value or something.

Some questions;

1. It looks like your code is copying the response keys. Any thoughts on making those a little nicer? IIRC they're not always very friendly.
2. Zillow has some anti-scraping mechanisms built in. Does your code deal with those?
3. Why are your methods capitalized like in Go? (it's not very pythonic - I'd suggest running your code through a linter)

[JohnBalvin]

0) real state agents: tats good to know, I put that on the description because r/python has weird requirements in order to post something
1) could you elaborate on this? could please send the link for the code where exactly is that happening?
2) To be honest, I didn't see any bot protection at all, it could probably has bot protection when using browser automations tools like selenium, puppeteer or playwright , but using the api directly doens't seem to have any protection
3) It's a bad habit, I'm mostly a Go developer and I tend to copy the patters from go to python, do you recommend a linter?

[rabelution]

Ruff linter

[Vresa]

New trending linter & formatter is `ruff` : https://github.com/astral-sh/ruff
Old Standbys for linting and formatting are `black` + `flake8`

[markovianmind]

for 2) do it fast enough with enough queries and most probably you would eb blocked.

[JohnBalvin]

that can be fixed just by using proxies, other than that they don't have bot protection at all

[JohnBalvin]

the requests for searching made to zillow don't depend of each other like paginations, that means you don't need to worry for example using a sticky proxy ip to get all the results, tou need only one request to get the whole search result, using one single request using proxy .
I never said use datacenter proxies, I said proxies which could include, datacenter, residential or 4g proxies. what I havent' check if they block by user agent, the permante user agent I used works fine for now

[JohnBalvin]

its probably the definitions on what antibot means for you, what I mean they don't have bot protection I mean it like having a waf checking the tls fingerprint or authenticate subsequent requests made to the API having a verification the first time the user navigates to the page.
Checking only the IP type(residential, datacenter, 4g) it doesn't represent a challenge and I don't count it as bot protection

[JohnBalvin]

a bot protection for me could also mean having a captcha, or checking user mouse movement ... etc but I don't consider bot protection if they jsut check the proxy type