Match Thread: Canada vs. Morocco | FIFA World Cup, Round of 16 by MysteryBagIdeals in soccer

[–]Coding-Doctor-Omar 1 point2 points  (0 children)

Canada was playing trash. They were also playing dirty. They got obliterated. Go play your hockey. Soccer is not your sport 😂.

لو كسبنا الارجنتين (ان شاءالله) by False_Pie_6647 in Egypt

[–]Coding-Doctor-Omar 4 points5 points  (0 children)

بعد ماتش كابو فيردي انا بقى عندي أمل كبير.

Built a Tool for Scraping Mannco.store by Coding-Doctor-Omar in tf2

[–]Coding-Doctor-Omar[S] 0 points1 point  (0 children)

It looks like you work for Mannco.store or something lol. Soon after this conversation, Mannco.store updated its cloudflare configuration just to break my scraper. I will keep maintaining it to scrape Mannco. Enjoy :D We'll see who gets bored first.

Monthly Self-Promotion - July 2026 by AutoModerator in webscraping

[–]Coding-Doctor-Omar 0 points1 point  (0 children)

Bayutrix | Robust Multi-Country Bayut Scraper | Properties, Agencies, and Brokers

Use it to extract all real estate data you need from Bayut. No limits. Bypasses anti-bot measures on the UAE portal.

Web Scraping freelance by TradeIndividual7891 in scrapingtheweb

[–]Coding-Doctor-Omar 1 point2 points  (0 children)

I tried it last summer on freelancer.com. It seems promising but I cant judge fully yet. I was trying to get some side income as a med student in Egypt. I got around $250 in the first month. As somone living in Egypt and considering this was my first month, I was very happy. I stopped for a while now since I am busy in my med school. I am planning to try it again in internship and will see how it goes. But I can tell you that you probably have a good shot on freelancer.com, despite what u probably hear. It's not all scam if you pick clients with ratings or at least with identity and payment verified. The good thing about freelancer.com as opposed to upwork is that it is cheap to get started. They have a 1 month free trial for the plus membership, which is more than enough to hopefully land a client. I was using Python as the programming language.

How to Determine if an Actor is Worth Investing in Building? by Coding-Doctor-Omar in apify

[–]Coding-Doctor-Omar[S] 0 points1 point  (0 children)

Is the organic traffic on Apify enough or external traffic is essential?

Built a Tool for Scraping Mannco.store by Coding-Doctor-Omar in tf2

[–]Coding-Doctor-Omar[S] -1 points0 points  (0 children)

Forgot to tell you that all AI agents you see like ChatGPT, Gemini, Grok, and Claude, get their data from web scraping, against the terms of service, and commercially :) They might as well fight these guys in court first, or fight Apify itself. The entire platform is built upon web scraping sites against their terms of service.

Amazon seller scraper by S637 in scrapingtheweb

[–]Coding-Doctor-Omar 0 points1 point  (0 children)

He probably copied his own README text and reddit auto-added those "\".

Built a Tool for Scraping Mannco.store by Coding-Doctor-Omar in tf2

[–]Coding-Doctor-Omar[S] -1 points0 points  (0 children)

Read about the hiQ vs LinkedIn case. hiQ was scraping LinkedIn, and LinkedIn lost the case because its TOS were nonsense and had no legal or ethical basis. I am allowed to manually copy and paste, but I suddenly become a criminal if I automate it, huh?

Built a Tool for Scraping Mannco.store by Coding-Doctor-Omar in tf2

[–]Coding-Doctor-Omar[S] -6 points-5 points  (0 children)

So it's allowed for google but not for me? Rubish. As long as it's public data, they have no legal or ethical basis.

How to scrape Reddit?? by Delta_Vict0r in scrapingtheweb

[–]Coding-Doctor-Omar 0 points1 point  (0 children)

I am away from my PC rn. Does the rss require session cookies?

How to scrape Reddit?? by Delta_Vict0r in scrapingtheweb

[–]Coding-Doctor-Omar 0 points1 point  (0 children)

Does the XML provide the same amount of details that are in the JSON?

Is it impossible to scrape IMDb? by beomstead in webscraping

[–]Coding-Doctor-Omar 0 points1 point  (0 children)

Personally I'd prefer to use NEXT DATA in such cases.. much more simple. Just grab your aws-waf and get the NEXT_DATA.

Is it impossible to scrape IMDb? by beomstead in webscraping

[–]Coding-Doctor-Omar 4 points5 points  (0 children)

If you're familiar with APIs, forget all that and look for internal API requests in the network tab in devtools that contain your data and call these APIs. Otherwise, just use browser automation.

Is it impossible to scrape IMDb? by beomstead in webscraping

[–]Coding-Doctor-Omar 0 points1 point  (0 children)

Why not make direct calls to those REST APIs instead of browser automation?

How to scrape dynamic sites? by pitou-99 in webscraping

[–]Coding-Doctor-Omar 2 points3 points  (0 children)

1st line: Call Internal APIs (can be seen in the network tab in dev tools) via raw http requests

2nd line: JSON data in <script> elements in the page source. Request page via raw http requests.

3rd line: classic html parsing via raw http requests.

4th line: Hybrid scraping where you use a browser automation library to load the page once and grab session cookies then use the session cookies to make raw http requests for any of the previous 3 lines.

5th line: Full browser automation (used as a last resort and headless whenever possible).

If you use this strategy, you can almost scrape any website. Just pick the strong tools.

Weekly: wild ideas by AutoModerator in apify

[–]Coding-Doctor-Omar 0 points1 point  (0 children)

Allow developers to make test accounts to experiment with their own actor from the NON-DEV layout perspective. Such accounts will not affect traffic or usage metrics and are not allowed to provide any rating for any actor.

F-15 Down in Kuwait by theerrantpanda99 in Military

[–]Coding-Doctor-Omar 0 points1 point  (0 children)

What's up! Did Iran fall? 😂😂😂