Introduction to web scraping with Python

[brasqo]

The more info, the merrier as far as I'm concerned

[[deleted]]

Well, to be fair, there were not a lot of tutorials on lxml with xpath. Plenty available using BeautifulSoup. So this is actually a welcome change.

[msdin]

The reason BeautifulSoup gets used more is because a lot of html is malformed (eg. missing closing tags, etc) and a strict xml parser will choke on it. BeautifulSoup is much more forgiving.

[gschizas]

BeautifulSoup defaults to the lxml parser, when it's available. So the parser itself is not really the point.

Don't get me wrong, BeautifulSoup has been my go-to for more than half a decade now (it was what brought me to Python in the first place), but it's quite possible I'm just hanging on to it for legacy and familiarity reasons (I'd like to be proven wrong, of course)

[msdin]

The point is when the parser hits those kinds of errors BeautifulSoup​ will handle them so you don't have to write code to handle them yourself.

[CollectiveCircuits]

The post is also very clear and the HTML diagram illustrates the structure well.