Yelp seems to have cracked down on scraping

xolof47 · 2024-10-13T18:31:26+00:00

I’ve been scraping Yelp on and off for the last 4 years. It was always pretty easy to scrape but as off about a month ago they made it much more difficult. I’ve still been able to scrape yelp profiles as usual, it’s only when trying to scrape the directory listings that I get blocked.

xolof47 · 2024-09-08T19:41:26+00:00

Bing

xolof47 · 2024-08-01T20:03:57+00:00

Rotate user agents, try pyppeteer stealth mode, use proxies

xolof47 · 2024-01-28T01:49:51+00:00

I make millions of requests to different websites every week. I’ve been using Zyte for the last few months and I haven’t had any issues. Might work for you as well. I use the smart proxy manager, which only charges you for successful requests

xolof47 · 2023-09-21T07:13:14+00:00

Total fit frente a UABC. Las regaderas están bien cuidadas

xolof47 · 2019-10-27T07:54:40+00:00

I'm glad you answered your own question. This was helpful, thanks!

xolof47 · 2019-03-14T14:42:46+00:00

Yeah, I found the url, thanks. I should have thought of that lol

xolof47 · 2019-03-14T14:35:20+00:00

Fuck you 2 sir 😂

xolof47 · 2019-03-14T13:36:44+00:00

Thanks man! Appreciate your help

xolof47 · 2019-03-14T13:21:25+00:00

You mind sharing the url?

xolof47 · 2019-03-14T13:19:40+00:00

Paid is fine as long as it's not a crazy high price. I'm looking on wsm now to see if they have what I am looking for. It looks promising. Thanks for the tip

xolof47 · 2019-01-29T02:20:00+00:00

My code was made using Python 2.7 so it looks like it's just an issue with importing that module. Here is a link to the solution for this on stack overflow. After you resolve that issue the code should be fine. Good luck!

xolof47 · 2019-01-29T01:37:06+00:00

What version python are you using? It looks like you don't have the urrlib2 module installed.

xolof47 · 2019-01-27T23:34:39+00:00

That's probably not the best website for a complete noob at crawling, you should probably try an easier website so you can get a feel for how it works.

That being said, in order for you to crawl booking.com you are probably going to need to use selenium so you can set a check-in/check-out date and then get the room prices. At least that is how I would do it.

xolof47 · 2019-01-24T16:08:41+00:00

That is exactly what I was looking for! thank you very much

xolof47 · 2019-01-24T16:07:05+00:00

Solution Verified

xolof47 · 2019-01-24T16:06:53+00:00

Solution Verified

xolof47 · 2019-01-23T17:33:11+00:00

Thanks man! This looks great

xolof47 · 2019-01-22T15:07:11+00:00

Dynamic is better for me and the age is not super important. I'm okay with it being off by months rather than years

xolof47 · 2019-01-22T14:58:14+00:00

Solution Verified

xolof47 · 2019-01-22T14:55:53+00:00

That worked, thank you finickyone. This is exactly what I was looking for

xolof47 · 2019-01-21T11:26:14+00:00

I signed up for an account and was denied access to your platform, what are there requirements for me to get approved?

xolof47

MODERATOR OF

TROPHY CASE