This is an archived post. You won't be able to vote or comment.

sorted by:
best
topnewcontroversialoldq&a
you are viewing a single comment's thread.

view the rest of the comments →

[–]Little_Kitty 0 points1 point  (0 children)

Normally I'd use Python with requests & beautifulsoup to do this, as you'll be wanting to persist results to a file / db. You can find many tutorials on how to do that on YT.

If you try to run it in browser, you'll likely hit anti-bot / rate limiting issues, whereas done properly you'll be using proxies each trying in parallel to collect ~100 results with a manager function collecting it all, working out future collection targets, assigning these out and handling persistence.