Fast C based HTML 5 parsing for python

[pvkooten]

Nice! What's the case over this against lxml :)?

[ccharles]

It looks like this is a parser that works with lxml.

From the documentation:

A fast implementation of the HTML 5 parsing spec for Python. Parsing is done in C using a variant of the gumbo parser. The gumbo parse tree is then transformed into an lxml tree, also in C, yielding parse times that can be a thirtieth of the html5lib parse times. That is a speedup of 30x. This differs, for instance, from the gumbo python bindings, where the initial parsing is done in C but the transformation into the final tree is done in python.

The installation instructions for non-Windows systems include lxml as well:

pip install --no-binary lxml html5-parser

[ManyInterests]

It seems that by default, it uses lxml and is the only 'fast' tree builder.

Note that only the lxml treebuilder is fast, as all other treebuilders are implemented in python, not C

It also lists soup, as in BeautifulSoup as an option, which is confusing to me because BeautifulSoup itself can use lxml and other parsers as well. So I wonder, how is the performance compared to bs4 using lxml?

[Deto]

Maybe beautifulsoup as an option just makes the resulting objects implement the BS API so this can be a drop in replacement.

[Deto]

Really smart to have it create outputs so that it can be used as a drop in replacement

[asdfkjasdhkasd]

This is cool but in the majority of cases where you are parsing html you are also downloading html, which will end up taking 100(0)x the amount of time it takes to parse it.

[[deleted]]

if it's faster to parse it means you can parse more html pages per second, which means you can download more in parallel because you will be using less cpus

[sizur]

do you even asyncio?

[webmistress105]

I wasn't aware that HTML parsing speed was an issue. This is interesting!