首页 > Web开发 > 详细

使用HttpClient抓取网站首页

时间:2016-01-06 15:29:08      阅读:310      评论:0      收藏:0      [点我收藏+]

HttpClient是Apache开发的第三方Java库,可以用来进行网络爬虫的开发,相关API的可以在http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/查看。

import java.io.BufferedReader;
import java.io.InputStreamReader;

import org.apache.http.client.methods.*;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;


public class pachong {
    public static void main(String[] args) throws Exception {
        String url = "http://www.baidu.com";
        
        CloseableHttpClient client = HttpClients.createDefault();
        HttpGet request = new HttpGet(url);
        
        CloseableHttpResponse response = client.execute(request);
        System.out.println("Response Code: " +
        response.getStatusLine().getStatusCode());
        
        BufferedReader rd = new BufferedReader(
            new InputStreamReader(response.getEntity().getContent()));
        String line = "";
        while((line = rd.readLine()) != null) {
            System.out.println(line);
        }
        response.close();
        client.close();
    }
}

 

使用HttpClient抓取网站首页

原文:http://www.cnblogs.com/finalboss1987/p/5105624.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!