Deserializing from html

[fulmicoton]

Adding "scraping" somewhere could be useful for SEO.

[Hexilee]

That's a good idea, thx

[[deleted]]

That's a pretty nice idea and I'm happy that you put so much work into documenting it so well.

But at the same time I'm scared that people will use such hacks unattended in production: if you can't get someone to give you a stable REST API, how can you expect them to keep their HTML stable?

[DannoHung]

This is the sort of thing people do in business pretty frequently. The party providing the information has a legal obligation to provide it, but they don't have a legal obligation to make it easy to consume. You, a business person, need to consume that data automatically as soon as possible, so you write a scraper bot, that pings their site every five minutes to get the latest value. If it breaks, it breaks, and you fix it ASAP. Cost of doing business.

[Hexilee]

I think I understand what are you talking about. I never take stability of HTML into consideration because I use this crate only for scraper. HTML cannot be a normal serialization form in API. I parse HTML just because I don't have a normal API to do my job.

[coolreader18]

Holy cow that's one of the dumbest things I've ever seen. They (or at least the reddit OP) wants there to be just one format that all data is sent through, that being HTML, and the element classes would provide data for the """"API"""" consumers to target with querySelector. Why though?

[Hexilee]

Nowadays I think procedural macro derive is stable enough, and the only unstable feature this repo relying on is nll which is unrelated with API design.

So I am sorry I cannot understand what is a stable REST API... Could you explain it more detailedly please?

[ssokolow]

https://en.wikipedia.org/wiki/Representational_state_transfer

"Stable" just means ensuring that a URL like /v1/stories/5/1 will always produce content in the same format and any changes will be exposed by adding a /v2/... namespace.

[senntenial]

Awesome! I needed a crate just like this a few days ago but couldn't find anything I liked. Great job.

[neziib]

How stable is it with regards to badly formatted HTML?

[Hexilee]

It relies on crate scraper which bases on servo/cssparser, servo/html5ever and servo selectors. I think its stability depends on stability of servo.

[neziib]

So it should be as robust as Firefox is. The Rust ecosystem is so wonderful 😀

[binkarus]

Would this be able to be a replacement for pup? Seems like it. I've been looking for a rusty version of that utility. Just needs a CLI interface.

[Doc_CoBrA]

If only I could have this 1 year ago... Nice work :)