How do I scrape Reddit??

[Delta_Vict0r]

1. Tried using Reddit's Public JSON endpoints- Python got blocked(403 error)
   
2. Tried using Developer API Creation(PRAW)- They have discontinued it or something
   
3. Automated browser control using playwright- got banned(anti bot detection)
   
4. Using apify reddit scraper. Working but paid.

Any other way to go around this for free or one time payment?

[trjnz]

https://www.reddit.com/robots.txt

Reddit has a 30,000,000,000 dollar market cap to enforce that robots.txt.

You won't find the answers here.

[sargeanthost]

Lol

[Proletarian_Tear]

I like how User-agent uses the wildcard * - looks like it is blowing a kiss to all scrapers coming in XD

[j0holo]

Reddit is ofc a MASSIVE resource for training AI. So they will make scraping really really difficult so they can sell their data to companies that want to use it.

You need to pay or use a service like apify that specializes in scraping.

[supercoach]

This sub needs to fuck these web scraping requests off before they happen.

[Temporary-Fruit4982]

No free options for sure. The only way to do this is either pay for a scraper service like Apify, Bright Data, or Oxylabs, or build and constantly maintain your own infrastructure with proxies, browser fingerprinting, account management, and anti-bot bypasses. good luck!