首页 > 其他 > 详细

scrapy中下载中间件异常处理

时间:2020-08-06 23:29:00      阅读:188      评论:0      收藏:0      [点我收藏+]

示例:

def process_exception(self, request, exception, spider):
        # Called when a download handler or a process_request()
        # (from other downloader middleware) raises an exception.

        # Must either:
        # - return None: continue processing this exception
        # - return a Response object: stops process_exception() chain
        # - return a Request object: stops process_exception() chain
        print(xxxx)
        # 不允许直接改url
        # request.url=‘https://www.baidu.com‘
        from scrapy import Request
        request=Request(url=https://www.baidu.com,callback=spider.parser)
        return request

 

scrapy中下载中间件异常处理

原文:https://www.cnblogs.com/baicai37/p/13449455.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!