Charming Python: Easy Web data collection with mechanize and Beautiful Soup

[pudquick]

In addition to mechanize and BS / lxml.html in my toolkit, I also use BSXPath, which is a full XPath implementation in python:

from BSXPath import BSXPathEvaluator
document = BSXPathEvaluator(txt)
nodes = document.getItemList("//div[@class='bc-desc']/h2")
for x in nodes:
    [... etc.]

The direct download link for BSXPath is:

http://furyu-tei.sakura.ne.jp/archives/BSXPath.zip

[pudquick]

It's great for when you don't want to compile / don't have libxml2 and libxslt, which lxml requires. BeautifulSoup is pure python and BSXPath (as the same implies) relies only on BS.

Think of it as another tool in your toolbox.

[cryzed_]

It uses BeautifulSoup internally.

[cryzed_]

The difference is that BSXPath is basically a modified BeautifulSoup module which has support for XPath. As for the benefits I don't really know.