百度搜索
import requests keyword = ‘Python‘ try: kv = {‘wd‘:keyword} r = requests.get(‘http://www.baidu.com/s‘,params=kv) print(r.request.url) r.raise_for_status() print(len(r.text)) print(r.text[100:500]) except: print(‘爬取失败‘)
https://wappass.baidu.com/static/captcha/tuxing.html?&ak=c27bbc89afca0463650ac9bde68ebe06&backurl=https%3A%2F%2Fwww.baidu.com%2Fs%3Fwd%3DPython&logid=10979694275783567205&signature=42cf12fd4df1ef835f2be81e3c2405da×tamp=1589703563 百度加了安全锁
1519
/title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black">
<meta name="viewport" content="width=device-width, user-scalable=no, initial-scale=1.0, minimum-scale=1.0, maximum-scale=1.0">
360搜索
import requests keyword = ‘Python‘ try: kv = {‘q‘:keyword} r = requests.get(‘http://www.so.com/s‘,params=kv) print(r.request.url) r.raise_for_status() print(len(r.text)) print(r.text[100:500]) except: print(‘爬取失败‘)
https://www.so.com/s?q=Python
547417
[endif]-->
<!--[if IE 8 ]><html class="ie8"><![endif]-->
<!--[if IE 9 ]><html class="ie9"><![endif]-->
<!--[if (gt IE 9)|!(IE)]><!--><html><!--<![endif]-->
<head>
<meta charset="utf-8">
<meta content="always" name="referrer">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>Python_360搜索</title>
<link rel="dns-prefetch" href="//p.ssl.qhimg.com"><link rel="dns-prefetch" href="//
原文：https://www.cnblogs.com/tingtin/p/12905701.html