大年初二,忙完家里一些事,顺带有人交流爬取豆瓣图书top250
1、构造urls列表 urls=[‘https://book.douban.com/top250?start={}‘.format(str(i) for i in range(0, 226, 25))]
2、模块 requests获取网页源代码 lxml 解析网页 xpath提取
3、提取信息
4、可以封装成函数 此处没有封装调用
python代码：
#coding:utf-8
import sys
reload(sys)
sys.setdefaultencoding(‘utf-8‘)
from lxml import etree
import requests
urls=[‘https://book.douban.com/top250?start={}‘.format(str(i) for i in range(0, 226, 25))]
for url in urls:
html=requests.get(url).content
selector=etree.HTML(html)
infos=selector.xpath(‘//tr[@class="item"]‘)
for info in infos:
book_name = info.xpath(‘td/div/a/@title‘)[0]
book_url = info.xpath(‘td/div/a/@href‘)[0]
published_infos = str(info.xpath(‘td/p/text()‘)[0])
splitlistinfos = published_infos.split(‘/‘)
#print splitlistinfos
published_date=str(splitlistinfos[-2])
#print published_date
price = str(splitlistinfos[-1])
#print price
rate = info.xpath(‘td/div/span[2]/text()‘)[0]
# comment_nums = info.xpath(‘td/div/span[3]/text()‘)[0]
# print comment_nums
comment_nums = info.xpath(‘td/div/span[3]/text()‘)[0].strip(‘(‘).strip().strip(‘)‘).strip().strip(‘人评价‘).strip() +‘人评价‘
introduceinfo = info.xpath(‘td/p/span/text()‘)
print book_name,book_url,published_date,price,rate,comment_nums,introduceinfo[0] if len(introduceinfo) > 0 else ‘‘
Python 2.7_利用xpath语法爬取豆瓣图书top250信息_20170129
原文：http://www.cnblogs.com/Mr-Cxy/p/6357570.html