下面是些处理这些站点的建议(tips):
COOKIES_ENABLED
DOWNLOAD_DELAY
http://scrapy-chs.readthedocs.org/zh_CN/latest/topics/practices.html
http://my.oschina.net/u/1024140/blog/188154
scrapy相关
原文:http://www.cnblogs.com/lixun/p/4565402.html