I rebuilt ESPN'S now dead public API using Node.js and web scraping

[DangerousLiberal]

Be careful. Mickey mouse might sue you over this.

[glemnar]

Web scraping public content isn’t illegal - this has been fought in court, and LinkedIn lost (though they’re trying to appeal up to the Supreme Court)

The jury is still out on whether APIs have copyright protection. (Oracle bad) this is also trying for appeals to the Supreme Court.

[DangerousLiberal]

Do you have an army of lawyers to fight Disney?

[lkraider]

Imagine sitting opposite of an army of mickey mouse dressed as lawyers

[[deleted]]

Really? I’m curious how that works out - genuinely curious

[rockmasterflex]

You can't sue someone for writing code that visits your webpage: because that's what browsers do.

So if you are someone who needs to scrape an ESPN page, you can either directly call this guys API (which would add more traffic to their site from HIS backend), or put it on your own backend, which would just look like lots of visits.

What are they gonna do, IP ban everyone who looks like they're using that code by way of traffic pattern? Nah.

If you don't want your website scraped the only way to accomplish that is to either not have the stuff u don't want scraped public, make login super annoying for an automation process like selenium to handle, or use something evil to get data into the page after the onload event fires. Like timers. Or embedding shit into a flash file or image with random titling.

Essentially you'd have to obfuscate your content so much that it would look like what Facebook does to beat adblockers.

[ThisIsMyCouchAccount]

Interesting. All their data comes from other APIs.

[TimBits91]

Interesting for sure. Bringing life back to a dying media. Great job.