[JAVA] [Updating] It was difficult to upgrade from httpclient 3.x series to 4.5, so I will write a summary of changes

I don't know what I've done, so I'll write down what I changed.

I was allowed to reference.

** Supports major interface changes from Java apache HttpClient 4.3 (@mychaelstyle) ** https://qiita.com/mychaelstyle/items/e02b3011d1e71bfa26c5 ** Apache httpclient CloseableHttpClient to HTTPS site Request with BASIC authentication (@sh_kawakami) ** https://qiita.com/sh_kawakami/items/bf6d1397851cccd134b2 ** It seems that the interface of httpclient has changed significantly from 4.3 (@sakito) ** https://qiita.com/sakito/items/6366015dbbc4a88d56fc What does setDefaultMaxPerRoute and setMaxTotal mean in HttpClient? Answer column for https://stackoverflow.com/questions/30689995/what-does-setdefaultmaxperroute-and-setmaxtotal-mean-in-httpclient

environment

JDK 1.8.162 httpclient 4.5.4

pom.xml

pom.xml


<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.4</version>
</dependency>

Nice Deprecated List

I've been looking around like I've never been helped by the Deprecated List

―― 3.x series https://hc.apache.org/httpclient-3.x/apidocs/deprecated-list.html ―― 4.x series http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/deprecated-list.html

Nice tutorial

http://hc.apache.org/httpcomponents-client-ga/tutorial/html/index.html

Nice Quick Guide

Since it is a guide when raising from 3.x to 4.0, it may change again when it reaches 4.5. http://debuguide.blogspot.jp/2013/01/quick-guide-for-migration-of-commons.html

List of useful constants

―― 3.x series https://hc.apache.org/httpclient-3.x/apidocs/constant-values.html ―― 4.x series http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/constant-values.html

Get rid of compilation errors

Package change

Change to standard API

A system that is completely different

The way to create an instance has changed in the first place

before.java


HttpClient client = new HttpClient();

Let's use Builder! It seems that. If you don't have to think about anything, below.

after.java


HttpClient client = HttpClients.createDefault();

However, it may not be very practical because you have to pass the connection settings at the timing of generation.

method class change

GetMethod→HttpGet PostMethod→HttpPost Parent class HttpMethod → HttpRequestBase Change to

Execution & Get Response

It used to be like this

before.java


PostMethod post = new PostMethod(url);
post.setRequestEntity(Set something);
int responseCode = client.executeMethod(post);
if (responseCode == 200) {
	Reader reader = new InputStreamReader(
			httpMethod.getResponseBodyAsStream(), httpMethod.getResponseCharSet());
	//Subsequent processing
} else {
  //Error handling
}

It's changed a lot and looks like this. But now that it's clear, it may be easier to understand.

after.java


HttpPost post = new HttpPost(url);
//Set the one that corresponds to the request body
post.setEntity(Set something);
HttpResponse response = client.execute(post);
StatusLine statusLine = response.getStatusLine();
int responseCode = statusLine.getStatusCode();
if (responseCode == HttpStatus.SC_OK) {
	Reader reader = new InputStreamReader(
			response.getEntity().getContent(), Charset.forName("UTF-8"));
	//Subsequent processing
} else {
	//Error handling
}

Around basic authentication connection

I used to do it like this.

before.java


Credentials credentials = new UsernamePasswordCredentials(this.username, this.password);
AuthScope scope = new AuthScope(host, port, this.realm);
client.getState().setCredentials(scope, credentials);

It seems that you should use CredentialsProvider. (Thanks to @sh_kawakami!)

after.java


CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(baseUrl.getHost(),baseUrl.getPort()),
        new UsernamePasswordCredentials(username, password));
//I will set it when creating an instance
HttpClient client = HttpClients.custom().setDefaultCredentialsProvider(credentialsProvider).build();

Interfaceing NameValuePair

It seems that the entity class is now BasicNameValuePair. It seems that it is just divided by "=", so I pray that the behavior will not change. But the setter is gone! !! !! It looks like it can only be set in the constructor. If you still want to use setter, you have to inherit NameValuePair and create your own class.

This and that that caused an error at runtime

The default HttpClient no longer sends Host headers

Let's add it ourselves

//host header
httpMethod.addHeader("Host", uri.getHost());

ConnectionClosedException occurs when Content-Length header and ResponseBody size do not match

If there is a Content-Length header in the response, it will occur if the size is different from the Body. It doesn't happen without the Content-Length header (though it feels like an RFC violation) For example, it occurs when the response is like this. Well, today I think that Content-Length is calculated automatically and the server returns it without permission, so I think that it rarely occurs, but for various reasons I do not trust the response header: sweat:

HTTP/1.1 200 OK
Content-Type: text/html; charset=Shift_JIS
Content-Length: 3000

<html>
<head><title>Hoge</title>
</head>
<body>test
</body>
</html>

org.apache.http.ConnectionClosedException: Premature end of Content-Length delimited message body (expected: 3000; received: 124
	at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:178)
	at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:135)
	at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:148)

The cause is here

java:org.apache.http.impl.io.ContentLengthInputStream.java


    /**
     * Does standard {@link InputStream#read(byte[], int, int)} behavior, but
     * also notifies the watcher when the contents have been consumed.
     *
     * @param b     The byte array to fill.
     * @param off   Start filling at this position.
     * @param len   The number of bytes to attempt to read.
     * @return The number of bytes read, or -1 if the end of content has been
     *  reached.
     *
     * @throws java.io.IOException Should an error occur on the wrapped stream.
     */
    @Override
    public int read (final byte[] b, final int off, final int len) throws java.io.IOException {
        if (closed) {
            throw new IOException("Attempted read from closed stream.");
        }

        if (pos >= contentLength) {
            return -1;
        }

        int chunk = len;
        if (pos + len > contentLength) {
            chunk = (int) (contentLength - pos);
        }
        final int count = this.in.read(b, off, chunk);
        if (count == -1 && pos < contentLength) {
            throw new ConnectionClosedException(
                    "Premature end of Content-Length delimited message body (expected: "
                    + contentLength + "; received: " + pos);
        }
        if (count > 0) {
            pos += count;
        }
        return count;
    }

I haven't found a solution so far.

Certificate and hostname do not match during SSL communication (including self-signed certificate) Error

javax.net.ssl.SSLPeerUnverifiedException: Certificate for <hostname> doesn't match any of the subject alternative names:

It seems that the host name is verified by default. The relevant part is here

java:org.apache.http.conn.ssl.SSLConnectionSocketFactory.java#verifyHostname


            if (!this.hostnameVerifier.verify(hostname, session)) {
                final Certificate[] certs = session.getPeerCertificates();
                final X509Certificate x509 = (X509Certificate) certs[0];
                final List<SubjectName> subjectAlts = DefaultHostnameVerifier.getSubjectAltNames(x509);
                throw new SSLPeerUnverifiedException("Certificate for <" + hostname + "> doesn't match any " +
                        "of the subject alternative names: " + subjectAlts);
            }

Since verify only needs to return true, it seems that an instance of the NoopHostnameVerifier class should be used. Disable SSL certificate verification and host name verification in Apache HttpComponents / Client Chapter 2. Connection management 2.7.4. Hostname verification In my case, I do not set SSLContext or SSLConnectionSocketFactory directly in httpclient, but implement it by setting it in ConnectionManager. At this point. So it looks like the following.

public static PoolingHttpClientConnectionManager createConnectionManager() {
    SSLContext sslContext = null;
	try {
		sslContext = new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy(){
			@Override
			public boolean isTrusted(X509Certificate[] chain, String authType) throws CertificateException {
				return true;
			}
		}
		).build();
	} catch (Exception e) {
		e.printStackTrace();
	}
	SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext, NoopHostnameVerifier.INSTANCE);
	Registry<ConnectionSocketFactory> registry =
			RegistryBuilder.<ConnectionSocketFactory>create()
					.register("http", PlainConnectionSocketFactory.getSocketFactory())
					.register("https", sslsf)
					.build();
	return new PoolingHttpClientConnectionManager(registry);
}

Other sites that I referred to

Apache HttpClient DefaultHttpRequestRetryHandler does not retry on ConnectTimeoutException http://kntmr.hatenablog.com/entry/2016/12/09/150615 Use of non-ascii credentials not working in httpclient 4.3.x https://stackoverflow.com/questions/27955067/use-of-non-ascii-credentials-not-working-in-httpclient-4-3-x

Recommended Posts

[Updating] It was difficult to upgrade from httpclient 3.x series to 4.5, so I will write a summary of changes
Rails was difficult, so I made something like a controller of Spring Framework to take a break
What I was addicted to when updating the PHP version of the development environment (Docker) from 7.2.11 to 7.4.x
I was able to obtain OCJP Silver SE 11 safely, so a summary
Summary of points I was worried about when migrating from java to kotlin
Summary of moss when updating from JMockit 1.4 to 1.30
I don't know, so I'm going to write a list (you don't have to read it)
kintone clone? I was quite addicted to launching OSS WebDB Extension with Lightsail + Docker, so make a note of it.