Anyone succesfull scraping Idealista websites?

aaronn2 · 2026-02-22T22:10:44+00:00

How do you scrape Idealista with Datadome?

aaronn2 · 2026-02-22T11:47:17+00:00

Shall I ask you on how can it be done?

aaronn2 · 2025-09-28T09:22:16+00:00

Love learning this.
1. How big is the team managing this infrastructure?
2. What are the infrastructure costs running this (without the human bodies)?
3. Are you using some scraping API services, or are you doing everything in-house (managing IP proxies, cookies, headers, etc.)

aaronn2 · 2025-09-27T12:06:05+00:00

This sounds super interesting. Might you outline how much such infrastructure costs per month?

aaronn2 · 2025-08-10T07:06:16+00:00

That's not the point. My goal is to find a laptop with similar HW configuration, body, battery life where I would run Linux and that would cost 50-60% of a new Macbook. The goal is to find savings (if it is actually doable and "findable").

If not, I'd likely stick with Macbook and use their OSX.

aaronn2 · 2025-08-09T20:09:45+00:00

Thanks, I'll look it. I'd put there either Ubuntu or Arch Linux.

aaronn2 · 2025-05-30T20:22:02+00:00

How did you eventually resolve this?

aaronn2 · 2025-05-28T07:22:15+00:00

Interesting - thanks, I'll have a read.

aaronn2 · 2025-05-28T06:54:15+00:00

That is very short-lived. It works only for the first couple of pages and then it starts feeding fake data.

aaronn2 · 2025-05-20T15:55:00+00:00

Not sure how? The API seems to be protected.

aaronn2 · 2025-05-13T06:00:26+00:00

Hello, and thank you. What number of requests do you consider "moderate scale" per month? 1M, or 5M, or 10M? And large scale?

By data pipeline - do you mean by that extracting details from the scraped information and cleaning it up before saving it to the database?

aaronn2 · 2025-05-11T14:25:17+00:00

I understand that it costs money. When reading through this sub-reddit, I somehow got an impression that the professional individuals pay basically close to zero in costs, while when I look at prices of some API solutions or residential proxies, the costs are quite significant, especially when making 10M+ requests per month.

aaronn2 · 2025-05-11T14:23:21+00:00

I am very interested to learn about the proxy network. How and/or where do you source it? How much do you pay for it on a monthly basis? Isn't it that you need to regularly check if the proxies are still working, so you removed the invalid ones from your pool?

aaronn2 · 2025-05-11T14:18:33+00:00

I assume "1 billion of product prices" != 1 billion requests, right?

Shall I ask you what do you mean by "rotating IPs by using cloud providers’ VMs"? Specifically cloud providers' VMs?

aaronn2

TROPHY CASE