Section Introduction:

In the previous section, we learned about HttpURLConnection. In this section, we move on to the second approach: HttpClient. Although it has been deprecated by Google, we can still use HttpClient to capture data in daily work, and it works even better when combined with Jsoup for parsing web pages! HttpClient is used to receive/send HTTP requests/responses, but it does not cache server responses, does not execute JavaScript code embedded in HTML pages, and does not perform any parsing or processing on page content! Let's begin this section!

1. HttpClient Usage Flow

Basic Flow：

2. HttpClient Usage Examples

1) Using HttpClient to Send a GET Request

Let's directly paste the simple code for sending a GET request:

public class MainActivity extends Activity implements OnClickListener {

    private Button btnGet;
    private WebView wView;
    public static final int SHOW_DATA = 0X123;
    private String detail = "";

    private Handler handler = new Handler() {
        public void handleMessage(Message msg) {
            if(msg.what == SHOW_DATA)
            {
                wView.loadDataWithBaseURL("",detail, "text/html","UTF-8","");
            }
        };
    };
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        initView();
        setView();
    }

    private void initView() {
        btnGet = (Button) findViewById(R.id.btnGet);
        wView = (WebView) findViewById(R.id.wView);
    }

    private void setView() {
        btnGet.setOnClickListener(this);
        wView.getSettings().setDomStorageEnabled(true);
    }
    @Override
    public void onClick(View v) {
        if (v.getId() == R.id.btnGet) {
            GetByHttpClient();
        }
    }
    private void GetByHttpClient() {
        new Thread()
        {
            public void run() 
            {
                    try {
                        HttpClient httpClient = new DefaultHttpClient();
                        HttpGet httpGet = new HttpGet("http://www.w3cschool.cc/python/python-tutorial.html");
                        HttpResponse httpResponse = httpClient.execute(httpGet);
                        if (httpResponse.getStatusLine().getStatusCode() == 200) {
                            HttpEntity entity = httpResponse.getEntity();
                            detail = EntityUtils.toString(entity, "utf-8");
                            handler.sendEmptyMessage(SHOW_DATA);
                        }
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
            };
        }.start();
    }

}

Screenshot of running result

In addition, if it is a GET request with parameters, we can put the parameters into a List collection, then URL-encode the parameters, and finally concatenate them with the URL:

List<BasicNameValuePair> params = new LinkedList<BasicNameValuePair>();  
params.add(new BasicNameValuePair("user", "猪小弟"));  
params.add(new BasicNameValuePair("pawd", "123"));
String param = URLEncodedUtils.format(params, "UTF-8"); 
HttpGet httpGet = new HttpGet("http://www.baidu.com"+"?"+param);

2) Using HttpClient to Send a POST Request

A POST request is slightly more complicated than a GET request. After creating the HttpPost object, store the parameters to be submitted in a NameValuePair collection, pass the parameters to UrlEncodedFormEntity, and finally complete it by calling setEntity(entity), then HttpClient.execute(HttpPost) will do. I won't write an example here; I haven't found a website that accepts POST for now, and I don't want to write a Servlet myself, so I'll just paste the core code directly.

Core Code:

private void PostByHttpClient(final String url)
{
    new Thread()
    {
        public void run() 
        {
            try{
                HttpClient httpClient = new DefaultHttpClient();
                HttpPost httpPost = new HttpPost(url);
                List<NameValuePair> params = new ArrayList<NameValuePair>();
                params.add(new BasicNameValuePair("user", "猪大哥"));
                params.add(new BasicNameValuePair("pawd", "123"));
                UrlEncodedFormEntity entity = new UrlEncodedFormEntity(params,"UTF-8");
                httpPost.setEntity(entity);
                HttpResponse httpResponse =  httpClient.execute(httpPost);
                if (httpResponse.getStatusLine().getStatusCode() == 200) {
                    HttpEntity entity2 = httpResponse.getEntity();
                    detail = EntityUtils.toString(entity2, "utf-8");
                    handler.sendEmptyMessage(SHOW_DATA);
                }
            }catch(Exception e){e.printStackTrace();}
        };
    }.start();
}

3. HttpClient Data Scraping Example (Educational Administration System Data Scraping)

Actually, there are many examples of using HttpClient. For example, I once used it to scrape students' course schedules from the school's educational administration system. This involves things like cookies and simulated login. When it comes to data scraping (crawlers), we usually use Jsoup to parse the scraped data. If you are interested, you can look up relevant materials yourself. Here, I'll paste the key code for fetching web pages from my graduation project app! You can get a sense of it:

HttpClient can obtain and set cookies through the following code:HttpResponse loginResponse = new DefaultHttpClient().execute(getLogin);Obtain Cookies:cookie = loginResponse.getFirstHeader("Set-Cookie").getValue();Attach Cookies When Making Requests:httpPost.setHeader("Cookie", cookie);

//获得链接,模拟登录的实现:
public int getConnect(String user, String key) throws Exception {
    // 先发送get请求 获取cookie值和__ViewState值
    HttpGet getLogin = new HttpGet(true_url);
    // 第一步:主要的HTML:
    String loginhtml = "";
    HttpResponse loginResponse = new DefaultHttpClient().execute(getLogin);
    if (loginResponse.getStatusLine().getStatusCode() == 200) {
        HttpEntity entity = loginResponse.getEntity();
        loginhtml = EntityUtils.toString(entity);
        // 获取响应的cookie值
        cookie = loginResponse.getFirstHeader("Set-Cookie").getValue();
        System.out.println("cookie= " + cookie);
    }

    // 第二步:模拟登录
    // 发送Post请求,禁止重定向
    HttpPost httpPost = new HttpPost(true_url);
    httpPost.getParams().setParameter(ClientPNames.HANDLE_REDIRECTS, false);

    // 设置Post提交的头信息的参数
    httpPost.setHeader("User-Agent",
            "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko");
    httpPost.setHeader("Referer", true_url);
    httpPost.setHeader("Cookie", cookie);

    // 设置请求数据
    List<NameValuePair> params = new ArrayList<NameValuePair>();

    params.add(new BasicNameValuePair("__VIEWSTATE",
            getViewState(loginhtml)));// __VIEWSTATE参数，如果变化可以动态抓取获取
    params.add(new BasicNameValuePair("Button1", ""));
    params.add(new BasicNameValuePair("hidPdrs", ""));
    params.add(new BasicNameValuePair("hidsc", ""));
    params.add(new BasicNameValuePair("lbLanguage", ""));
    params.add(new BasicNameValuePair("RadioButtonList1", "%D1%A7%C9%FA"));
    params.add(new BasicNameValuePair("txtUserName", user));
    params.add(new BasicNameValuePair("TextBox2", key));
    params.add(new BasicNameValuePair("txtSecretCode", "")); // ( ╯□╰ )逗比正方,竟然不需要验证码

    // 设置编码方式,响应请求,获取响应状态码:
    httpPost.setEntity(new UrlEncodedFormEntity(params, "gb2312"));
    HttpResponse response = new DefaultHttpClient().execute(httpPost);
    int Status = response.getStatusLine().getStatusCode();
    if(Status == 200)return Status;
    System.out.println("Status= " + Status);

    // 重定向状态码为302
    if (Status == 302 || Status == 301) {
        // 获取头部信息中Location的值
        location = response.getFirstHeader("Location").getValue();
        System.out.println(location);
        // 第三步:获取管理信息的主页面
        // Get请求
        HttpGet httpGet = new HttpGet(ip_url + location);// 带上location地址访问
        httpGet.setHeader("Referer", true_url);
        httpGet.setHeader("Cookie", cookie);

        // 主页的html
        mainhtml = "";
        HttpResponse httpResponseget = new DefaultHttpClient()
                .execute(httpGet);
        if (httpResponseget.getStatusLine().getStatusCode() == 200) {
            HttpEntity entity = httpResponseget.getEntity();
            mainhtml = EntityUtils.toString(entity);
        }

    }
    return Status;
}

4. Using HttpPut to Send a PUT Request

The example code is as follows:：

public static int PutActCode(String actCode, String licPlate, Context mContext) {
    int resp = 0;
    String cookie = (String) SPUtils.get(mContext, "session", "");
    HttpPut httpPut = new HttpPut(PUTACKCODE_URL);
    httpPut.setHeader("Cookie", cookie);
    try {

        List<NameValuePair> params = new ArrayList<NameValuePair>();
        params.add(new BasicNameValuePair("activation_code", actCode));
        params.add(new BasicNameValuePair("license_plate", licPlate));
        httpPut.setEntity(new UrlEncodedFormEntity(params, "UTF-8"));
        HttpResponse course_response = new DefaultHttpClient().execute(httpPut);
        if (course_response.getStatusLine().getStatusCode() == 200) {
            HttpEntity entity2 = course_response.getEntity();
            JSONObject jObject = new JSONObject(EntityUtils.toString(entity2));
            resp = Integer.parseInt(jObject.getString("status_code"));
            return resp;
        }
    } catch (Exception e) {
        e.printStackTrace();
    }
    return resp;
}

Section Summary:

All right, this section on the second HTTP request method in Android, HttpClient, ends here. In the next section, we will start learning about XML and JSON parsing. That's all for this section, thanks~