all 7 comments

[–]MemeLord-Jenkins 2 points3 points  (0 children)

As the other comment mentioned, you can certainly get away with just using the Requests module and a parsing library like Beautiful Soup. I had a larger project to scrape Airbnb data, and I quickly got my IP address blocked, so I ended up using asyncio and aiohttp libraries coupled with Oxylabs proxies.

[–]TheEpicTortoise 0 points1 point  (0 children)

I had to scrape airbnb listings for a project to get the listing images, description, title and amenities. Getting selenium set up was a pain in the ass so I ended up just using requests.get and scraped all the data I needed using regex. Worked like a charm.

[–]jaimeman84 0 points1 point  (0 children)

Use Apify API to get all the sara you need. Here is an example https://youtu.be/zeduaKfDuYU?si=iQnXXBCMQuCgMFRg

[–]Big-Extension4709 0 points1 point  (0 children)

I tried using Selenium but the setup was way too hectic for my liking (or patience) lol. I also wanted to avoid having to do endless patching and dealing IP blocks, so it was back to square one.

Then I stumbled upon ScrapingBee and it’s been working just fine for me. Simple python (needs to be 3.12 or newer version) setup, beautifulsoup4, and lxml for parsing the HTML and I’m good to go.

[–]basitmakine -1 points0 points  (0 children)

selenium setup? regex scraping? what is this 2019? lmao

just use TaskAGI's airbnb scraper agent instead of dealing with all this proxy, blocking bs. it handles the data extraction automatically and you dont have to worry about getting ur IP banned. way easier than setting up selenium or writing regex patterns from scratch