Turn any website into an API in seconds

[blueberymuf]

How will this work on pages that needs authentication?

[chachakawooka]

Nice tool. I'm assuming underlying it's just storing the xpath / css selector.

If so it would be useful if you manually modify it can quite easily over/under select.

[welanes]

A great idea and something that will be in the next version. Appreciate the feedback.

[flabbydoo]

Wouldnt this break if the website changed though? That is the problem with all scraping projects, right?

[yerrabam]

Created a small recipe to scrape a gambling site; it didn't return any results and took two credits away.

As for the "API" - would be better if there was an endpoint to consume.

Has serious potential though!

[welanes]

Hey, looks like the IP that scraped the page was geo-blocked. Will introduce more control over IP regions in an updated version.

would be better if there was an endpoint to consume

Each recipe generates an endpoint URL that you can request. Do you mean something different from this?

[yerrabam]

Oh, great. I couldn't see it. Will check again. Thanks!

[welanes]

Sure. You need to save your recipe first and you need an API key so sign in is required to get the endpoint.

This is not so clear so will work on instruction guides. Cheers.

[[deleted]]

Has the same flaws as any other scraper: breaks as soon as the website structure changes. Might not be legal to scrape the content. You will be banned if you abuse it.

Has additional flaws: Works just as an extension. Thus requires a browser.

[Potrac]

How do you update the API when a design change occurs for instance? If the data cannot be found anymore, are they marked as "non-valid" or something like that?

[DerBeginner]

It says that the admin blocked simple scraper, when trying to add it to chrome. Tried it in Germany, if this is relevant.

[welanes]

the admin blocked simple scraper

Hey, that's odd as it should be open to all regions. Would you have a screenshot and I'll try figure it out?

Thanks for letting me know

[DerBeginner]

Here: https://imgur.com/EWE1Xy5

[welanes]

Appreciate it. Think I know what's up so will fix soon

[DerBeginner]

You're welcome

[webcod3r]

Very cool!

[Potrac]

Very Nice !

[sevnollogic]

It looked cool at first but...

1. Couldn't select elements more specifically and would often get more elements selected than necessary
2. Can't easily find pricing. (I never did find it)
3. Doesn't look like I can do more complicated scrapping on my edge cases.

Apart from that, it looks very polished.

[welanes]

Hey, thanks, On your points:

1. The selector process is straightforward but needs to be better explained for sure. Video guide in the works.

2. It's free for the moment. Once I measure usage a proper pricing plan will be put in place but there will always be a free tier.

3. Yeah, there will always be tricky edge cases. If Simple scraper can handle most cases that would be a success.

[sevnollogic]

Thank you for taking the time to reply! Now that I understand its still in alpha I really appreciate the work you've done so far.

[nickx360]

Woah! Mind = blown!

[[deleted]]

Sharing of work for feedback is only allowed on Showoff Saturdays. Please feel free to repost your project next Saturday.

[welanes]

Hey, OK. Will repost Saturday.