很简单,教你一招:
这是你的html文件,我命名为test.html
然后用xpath匹配:
import lxml.etree
html=lxml.etree.parse("test.html")
res=html.xpath("//a[@heaf='baidu.com']")
info=res[0].xpath('string(.)')
print(info)
运行结果:
我要取出mrlevo520的内容,怎么取呢,很多方法,bs4也可以,正则也可以,动态selenium也可以,这次我想尝试用xpath来做,一则是为了和selenium接轨,xpath的确很强大,二来是firefox提供firebug插件,可以直接定位你需要内容的标签,一步到位简直完美,不多说,上程序。
import urllib2
from lxml import etree
crawl_url = "http://www.jianshu.com/p/e2c4ebd2eeb3"
req = urllib2.Request(crawl_url)
req.add_header('User-Agent','Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36')
response = urllib2.urlopen(req)
html = response.read()
selector = etree.HTML(html)
# 核心部分
bloger = selector.xpath("//a[@class='author-name blue-link']")
info = bloger[0].xp