src/docbkx/advanced.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE preface PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
                 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
<!--
    ====================================================================
    Licensed to the Apache Software Foundation (ASF) under one
    or more contributor license agreements.  See the NOTICE file
    distributed with this work for additional information
    regarding copyright ownership.  The ASF licenses this file
    to you under the Apache License, Version 2.0 (the
    "License"); you may not use this file except in compliance
    with the License.  You may obtain a copy of the License at
    
    http://www.apache.org/licenses/LICENSE-2.0
    
    Unless required by applicable law or agreed to in writing,
    software distributed under the License is distributed on an
    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    KIND, either express or implied.  See the License for the
    specific language governing permissions and limitations
    under the License.
    ====================================================================
-->
<chapter id="advanced">
    <title>Advanced topics</title>
    <section>
        <title>Custom client connections</title>
        <para>In certain situations it may be necessary to customize the way HTTP messages get
            transmitted across the wire beyond what is possible using HTTP parameters in
            order to be able to deal non-standard, non-compliant behaviours. For instance, for web
            crawlers it may be necessary to force HttpClient into accepting malformed response heads
            in order to salvage the content of the messages. </para>
        <para>Usually the process of plugging in a custom message parser or a custom connection
            implementation involves several steps:</para>
        <itemizedlist>
            <listitem>
                <para>Provide a custom <interfacename>LineParser</interfacename> /
                        <interfacename>LineFormatter</interfacename> interface implementation.
                    Implement message parsing / formatting logic as required.</para>
                <programlisting><![CDATA[
class MyLineParser extends BasicLineParser {

    @Override
    public Header parseHeader(
            CharArrayBuffer buffer) throws ParseException {
        try {
            return super.parseHeader(buffer);
        } catch (ParseException ex) {
            // Suppress ParseException exception
            return new BasicHeader(buffer.toString(), null);
        }
    }

}
]]></programlisting>
            </listitem>
            <listitem>
                <para>Provide a custom <interfacename>HttpConnectionFactory</interfacename>
                    implementation. Replace default request writer and / or response parser
                    with custom ones as required. </para>
                <programlisting><![CDATA[
HttpConnectionFactory<HttpRoute, ManagedHttpClientConnection> connFactory =
        new ManagedHttpClientConnectionFactory(
            new DefaultHttpRequestWriterFactory(),
            new DefaultHttpResponseParserFactory(
                    new MyLineParser(), new DefaultHttpResponseFactory()));
]]></programlisting>
            </listitem>
            <listitem>
                <para>Configure HttpClient to use the custom connection factory.</para>
                <programlisting><![CDATA[
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager(
    connFactory);
CloseableHttpClient httpclient = HttpClients.custom()
        .setConnectionManager(cm)
        .build();
]]></programlisting>
            </listitem>
        </itemizedlist>
    </section>
    <section id="stateful_conn">
        <title>Stateful HTTP connections</title>
        <para>While HTTP specification assumes that session state information is always embedded in
            HTTP messages in the form of HTTP cookies and therefore HTTP connections are always
            stateless, this assumption does not always hold true in real life. There are cases when
            HTTP connections are created with a particular user identity or within a particular
            security context and therefore cannot be shared with other users and can be reused by
            the same user only. Examples of such stateful HTTP connections are
                <literal>NTLM</literal> authenticated connections and SSL connections with client
            certificate authentication.</para>
        <section>
            <title>User token handler</title>
            <para>HttpClient relies on <interfacename>UserTokenHandler</interfacename> interface to
                determine if the given execution context is user specific or not. The token object
                returned by this handler is expected to uniquely identify the current user if the
                context is user specific or to be null if the context does not contain any resources
                or details specific to the current user. The user token will be used to ensure that
                user specific resources will not be shared with or reused by other users.</para>
            <para>The default implementation of the <interfacename>UserTokenHandler</interfacename>
                interface uses an instance of Principal class to represent a state object for HTTP
                connections, if it can be obtained from the given execution context.
                    <classname>DefaultUserTokenHandler</classname> will use the user principal of
                connection based authentication schemes such as <literal>NTLM</literal> or that of
                the SSL session with client authentication turned on. If both are unavailable, null
                token will be returned.</para>
            <programlisting><![CDATA[
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpClientContext context = HttpClientContext.create();
HttpGet httpget = new HttpGet("http://localhost:8080/");
CloseableHttpResponse response = httpclient.execute(httpget, context);
try {
    Principal principal = context.getUserToken(Principal.class);
    System.out.println(principal);
} finally {
    response.close();
}
]]></programlisting>
            <para>Users can provide a custom implementation if the default one does not satisfy
                their needs:</para>
            <programlisting><![CDATA[
UserTokenHandler userTokenHandler = new UserTokenHandler() {

    public Object getUserToken(HttpContext context) {
        return context.getAttribute("my-token");
    }

};
CloseableHttpClient httpclient = HttpClients.custom()
        .setUserTokenHandler(userTokenHandler)
        .build();
]]></programlisting>
        </section>
        <section>
            <title>Persistent stateful connections</title>
            <para>Please note that a persistent connection that carries a state object can be reused
                only if the same state object is bound to the execution context when requests
                are executed. So, it is really important to ensure the either same context is
                reused for execution of subsequent HTTP requests by the same user or the user
                token is bound to the context prior to request execution.</para>
            <programlisting><![CDATA[
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpClientContext context1 = HttpClientContext.create();
HttpGet httpget1 = new HttpGet("http://localhost:8080/");
CloseableHttpResponse response1 = httpclient.execute(httpget1, context1);
try {
    HttpEntity entity1 = response1.getEntity();
} finally {
    response1.close();
}
Principal principal = context1.getUserToken(Principal.class);

HttpClientContext context2 = HttpClientContext.create();
context2.setUserToken(principal);
HttpGet httpget2 = new HttpGet("http://localhost:8080/");
CloseableHttpResponse response2 = httpclient.execute(httpget2, context2);
try {
    HttpEntity entity2 = response2.getEntity();
} finally {
    response2.close();
}
]]></programlisting>
        </section>
    </section>
    <section>
        <title>Using the FutureRequestExecutionService</title>

        <para>Using the FutureRequestExecutionService, you can schedule http calls and treat the response
            as a Future. This is useful when e.g. making multiple calls to a web service. The advantage of using
            the FutureRequestExecutionService is that you can use multiple threads to schedule requests concurrently, set timeouts on
            the tasks, or cancel them when a response is no longer necessary.
        </para>

        <para>FutureRequestExecutionService wraps the request with a HttpRequestFutureTask, which extends FutureTask. This
        class allows you to cancel the task as well as keep track of various metrics such as request duration.</para>

        <section>
            <title>Creating the FutureRequestExecutionService</title>
            <para>The constructor for the futureRequestExecutionService takes any existing httpClient instance and an ExecutorService
            instance. When configuring both, it is important to align the maximum number of connections with the number of threads
            you are going to use. When there are more threads than connections, the connections may start timing out because there are no
            available connections. When there are more connections than threads, the futureRequestExecutionService will not use all of them</para>

            <programlisting><![CDATA[
HttpClient httpClient = HttpClientBuilder.create().setMaxConnPerRoute(5).build();
ExecutorService executorService = Executors.newFixedThreadPool(5);
FutureRequestExecutionService futureRequestExecutionService =
    new FutureRequestExecutionService(httpClient, executorService);
]]></programlisting>
        </section>

        <section>
            <title>Scheduling requests</title>
            <para>To schedule a request, simply provide a HttpUriRequest, HttpContext, and a ResponseHandler. Because
                the request is processed by the executor service, a ResponseHandler is mandatory.</para>

            <programlisting><![CDATA[
private final class OkidokiHandler implements ResponseHandler<Boolean> {
    public Boolean handleResponse(
            final HttpResponse response) throws ClientProtocolException, IOException {
        return response.getStatusLine().getStatusCode() == 200;
    }
}

HttpRequestFutureTask<Boolean> task = futureRequestExecutionService.execute(
    new HttpGet("http://www.google.com"), HttpClientContext.create(),
    new OkidokiHandler());
// blocks until the request complete and then returns true if you can connect to Google
boolean ok=task.get();
]]></programlisting>
        </section>
        <section>
            <title>Canceling tasks</title>
            <para>Scheduled tasks may be cancelled. If the task is not yet executing but merely queued for execution, it simply will never execute. If it is executing and the mayInterruptIfRunning parameter is set to true, abort() will be called on the request; otherwise the response will simply be ignored but the request will be allowed to complete normally. Any subsequent calls to task.get() will fail with an IllegalStateException. It should be noticed that canceling tasks merely frees up the client side resources. The request may actually be handled normally on the server side. </para>
            <programlisting><![CDATA[
task.cancel(true)
task.get() // throws an Exception
]]></programlisting>
        </section>
        <section>
            <title>Callbacks</title>
            <para>Instead of manually calling task.get(), you can also use a FutureCallback instance that gets callbacks when the request completes. This is the
                same interface as is used in HttpAsyncClient</para>
                <programlisting><![CDATA[

private final class MyCallback implements FutureCallback<Boolean> {

    public void failed(final Exception ex) {
        // do something
    }

    public void completed(final Boolean result) {
        // do something
    }

    public void cancelled() {
        // do something
    }
}

HttpRequestFutureTask<Boolean> task = futureRequestExecutionService.execute(
    new HttpGet("http://www.google.com"), HttpClientContext.create(),
    new OkidokiHandler(), new MyCallback());
]]></programlisting>
        </section>
        <section>
            <title>Metrics</title>
            <para>FutureRequestExecutionService is typically used in applications that make large amounts of
                web service calls. To facilitate e.g. monitoring or configuration tuning, the FutureRequestExecutionService keeps
                track of several metrics.</para>
            <para>Each HttpRequestFutureTask provides methods to get the time the task was scheduled,
                started, and ended. Additionally, request and task duration are available as well. These
                metrics are aggregated in the FutureRequestExecutionService in a FutureRequestExecutionMetrics
                instance that may be accessed through FutureRequestExecutionService.metrics().</para>
            <programlisting><![CDATA[
task.scheduledTime() // returns the timestamp the task was scheduled
task.startedTime() // returns the timestamp when the task was started
task.endedTime() // returns the timestamp when the task was done executing
task.requestDuration // returns the duration of the http request
task.taskDuration // returns the duration of the task from the moment it was scheduled

FutureRequestExecutionMetrics metrics = futureRequestExecutionService.metrics()
metrics.getActiveConnectionCount() // currently active connections
metrics.getScheduledConnectionCount(); // currently scheduled connections
metrics.getSuccessfulConnectionCount(); // total number of successful requests
metrics.getSuccessfulConnectionAverageDuration(); // average request duration
metrics.getFailedConnectionCount(); // total number of failed tasks
metrics.getFailedConnectionAverageDuration(); // average duration of failed tasks
metrics.getTaskCount(); // total number of tasks scheduled
metrics.getRequestCount(); // total number of requests
metrics.getRequestAverageDuration(); // average request duration
metrics.getTaskAverageDuration(); // average task duration
]]></programlisting>
        </section>
    </section>
</chapter>