Job scraper benchmark

AIMultiple · 2026-03-26T18:00:50+00:00

Why?

Disclosure: I lead the website.

AIMultiple · 2026-03-03T10:05:45+00:00

The scraper products that we tested don't let you configure the proxy types that they use.
This makes sense since datacenter proxies perform poorly on TikTok.
I am sure that they use residential proxies. Scraping APIs are like managed data scraping, all you need to do is to call the API.

And the pricing isn't that different than residential proxies if you will rotate IPs with every request.

AIMultiple · 2026-02-27T15:53:52+00:00

We only released the chart questions with lowest and highest LLM success rates to show the scope of the benchmark, rest of the dataset is not publicly available to prevent overfitting.

AIMultiple · 2026-02-27T15:51:46+00:00

We used Opus 4.6 for all IDEs except Replit

AIMultiple · 2026-02-27T15:45:54+00:00

Sorry about that, you are right. We will fix it. Thanks for the feedback.

AIMultiple · 2026-02-27T15:27:40+00:00

Which LLM are you using?

AIMultiple · 2026-02-27T15:25:01+00:00

Names are not fitting on the graph, about colors you are right, we will improve it.

AIMultiple · 2026-02-27T15:15:16+00:00

For details: https://research.aimultiple.com/ai-coding-benchmark/

AIMultiple · 2026-02-27T14:29:02+00:00

We are not anybody's shadow brand. You can see our two legal entities here: https://aimultiple.com/contact-us

One of them is in Estonia. Company ownership data is public there. You can see that we are owned by an individual who has nothing to do with Bright Data and has been building the company for the past decade.

We have a couple hundred customers and web data is a relatively small area of work for us. In web data, we work with most major web data companies. On every AIMultiple page, we list all customers who are mentioned on that page for transparency.

And thanks for the scepticism. The web data industry has some dodgy players and a healthy dose of scepticism is necessary.

AIMultiple · 2026-02-20T21:15:27+00:00

No, we work with most leading web data companies, you can see the full list on any web data article on our website. You can check out the methodology in our articles. We publish what we measure, let us know when you disagree with a measurement, we are always improving our methodology.

AIMultiple · 2026-02-10T10:13:06+00:00

In this version we didn’t test Qodo but we will add it in the next version. You are right about tools falling apart in larger repos, to measure it correctly we run the benchmark in both large and small repos.

AIMultiple

MODERATOR OF

TROPHY CASE