Apache HttpClient tutorial

Background

Assuming you know what REST or Restful APIs (GET, POST etc) are in this post we will cover how can we make these REST calls with Java program.  As the title of the post suggests the library that we are going to use is  - Apache HttpClient. 

To test the same REST calls we can use the CURL command line utility. You can learn how to do so from my previous posts on CURL

1. Download and Install cURL on Windows 
2. Using HTTP POST and GET using cURL

Dependency

We need to include the Apache HTTP client library in the code. I always use Ivy as a dependency manager for my java programming.

 My Ivy file looks like - 

<ivy-module version="2.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd">
    <info
        organisation="OpenSourceForGeeks"
        module="Demos"
        status="integration">
    </info>
    
    
    <dependencies>
        <dependency org="org.apache.httpcomponents" name="httpclient" rev="4.3.1"/>        
    </dependencies>
    
</ivy-module>

Note : Whenever you want to add any dependency in any dependency Manager like Maven, Ivy, gradle etc you can search the library in Maven Repository.

Code

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.util.ArrayList;
import java.util.List;

import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;


/**
 * 
 * @author athakur
 *
 */
public class HttpClientDemo
{
    
    private static final String USER_AGENT = "Mozilla/5.0";
    private static final int TIMEOUT = 5000;
    
    public static void main(String args[])
    {
        
        HttpClient httpClient = new DefaultHttpClient();
        
        HttpClientDemo httpClientDemo = new HttpClientDemo();
        HttpParams httpParams = httpClient.getParams();
        httpParams.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, TIMEOUT);
        httpParams.setParameter(CoreConnectionPNames.SO_TIMEOUT, TIMEOUT);
        
        String getResponse = httpClientDemo.performGetRequest(httpClient, "http://www.google.co.in");
        System.out.println("GET Response : " + getResponse);
        String postResponse = httpClientDemo.performPostRequest(httpClient, "http://www.google.co.in");
        System.out.println("GET Response : " + postResponse);
    }
    

    private String performGetRequest(HttpClient httpClient, String url)
    {
        HttpGet httprequest = new HttpGet(url);
        httprequest.addHeader("User-Agent", USER_AGENT);

        
        try {
            System.out.println("Performing GET request on + " + url);
            HttpResponse httpResponse = httpClient.execute(httprequest);
            System.out.println("Status code : " + httpResponse.getStatusLine().getStatusCode());
            
            BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent()));

            StringBuffer requestResult = new StringBuffer();
            String line = "";
            while ((line = reader.readLine()) != null) {
                requestResult.append(line);
            }
            
            return requestResult.toString();
      
        }
        catch (IOException e) {
            System.out.println("Exception occurred while performing GET request");
            e.printStackTrace();
            return null;
        }
        finally {
            httprequest.releaseConnection();
        }
    }
    
    private String performPostRequest(HttpClient httpClient, String url)
    {
        HttpPost httprequest = new HttpPost(url);
        httprequest.addHeader("User-Agent", USER_AGENT);
        
        List<NameValuePair> urlPostParameters = new ArrayList<NameValuePair>();
        urlPostParameters.add(new BasicNameValuePair("key", "value"));
     
        try {
            httprequest.setEntity(new UrlEncodedFormEntity(urlPostParameters));
        }
        catch (UnsupportedEncodingException e1) {
            // TODO Auto-generated catch block
            System.out.println("Encoding not supported");
            e1.printStackTrace();
            return null;
        }

        
        try {
            System.out.println("Performing POST request on + " + url);
            HttpResponse httpResponse = httpClient.execute(httprequest);
            System.out.println("Status code : " + httpResponse.getStatusLine().getStatusCode());
            
            BufferedReader reader = new BufferedReader(new InputStreamReader(httpResponse.getEntity().getContent()));

            StringBuffer requestResult = new StringBuffer();
            String line = "";
            while ((line = reader.readLine()) != null) {
                requestResult.append(line);
            }
            
            return requestResult.toString();
      
        }
        catch (IOException e) {
            System.out.println("Exception occurred while performing POST request");
            e.printStackTrace();
            return null;
        }
        finally {
            httprequest.releaseConnection();
        }
    }
    
    
}

Output

Performing GET request on + http://www.google.co.in
Status code : 200
GET Response : <!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en-IN">...
Performing POST request on + http://www.google.co.in
Status code : 405
GET Response : <!DOCTYPE html><html lang=en>  .......

Note :  POST call will not return valid output (200 status code). We got 405 (method not allowed) for post request because POST is not a valid method type for URL http://www.google.com. It only accepts a GET request.

Setting Timeout

Just for the record - 

HttpParams httpParams = httpClient.getParams();
httpParams.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, TIMEOUT);
httpParams.setParameter(CoreConnectionPNames.SO_TIMEOUT, TIMEOUT);

is alternate way to do

HttpParams httpParams = httpClient.getParams();
HttpConnectionParams.setConnectionTimeout(httpParams, TIMEOUT);
HttpConnectionParams.setSoTimeout(httpParams, TIMEOUT);
 

With new APIs introduced in version 4.3 we can do

RequestConfig requestConfig = RequestConfig.custom()
  .setConnectTimeout(TIMEOUT)
  .setConnectionRequestTimeout(TIMEOUT)
  .setSocketTimeout(TIMEOUT).build();
CloseableHttpClient client =
  HttpClientBuilder.create().setDefaultRequestConfig(requestConfig).build();

Understanding Timeouts - 

• Connection Timeout – Time to establish the connection with the remote host.
• Socket Timeout  – Time waiting for response of a request – after the connection is established.
• Connection Manager Timeout  – Time to wait for a connection from the connection manager/pool .

Refer to following diagram to understand how REST APIs work -

Related Links

• Apache HTTP client tutorial
• How to send HTTP request GET/POST in Java
• What is the difference between the setConnectionTimeout , setSoTimeout and “http.connection-manager.timeout” in apache HttpClient API