The AWS API can take advantage of IAM authentication by signing your request with AWS Signature V4. Normally, you don't need to be very aware of the Signature V4 specifications by using the AWS SDK, but for various reasons, I had the opportunity to implement Signature V4 on my own, so I will leave a note.
It is implemented in Java 11 and the HTTP client uses Apache HttpComponents Client.
The code is cut and pasted and processed, so it is a little doubtful that it will work properly.
For the sake of clarity, here is a sample request when not signing. This is a sample search request using Elasticsearch API. Since this sample uses the POST method, the query parameters are written in HTTP body.
We will sign this request with AWS Signature V4 and modify it so that it will only receive the API from authenticated users.
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.HttpClientBuilder;
void reqFunc(){
//Request generation
URI uri = new URI(AWS_ELASTICSEARCH_URL + "/index/hoge/_search?");
HttpPost post = new HttpPost(uri);
post.setHeader("Content-Type", "application/json");
post.setEntity(new StringEntity( "Query string(abridgement)", "UTF-8"));
//Executing a request with an HTTP client
HttpClient client = HttpClientBuilder.create().build();
CloseableHttpResponse res = (CloseableHttpResponse) client.execute(post);
// ...Processing for response...
}
AWS Signature V4 is available on Amazon's website. I got the information by following the URL below. https://docs.aws.amazon.com/ja_jp/general/latest/gr/signature-version-4.html
In addition, the access key ID and secret access key acquisition method required for authentication are obtained from the URL below. https://docs.aws.amazon.com/ja_jp/general/latest/gr/aws-sec-cred-types.html
AWS Signature V4 generates a hash using the request information itself and the access key ID / secret access key issued in advance, and attaches it to the request header. I hashed it over and over again, and I didn't know how far the target of the hash was, so I made a trial and error.
First, in order to get the information of the request itself to be sent, prepare the HttpRequestInterceptor class and the HTTP client that sandwiches it. By inserting this class when generating an HTTP client, you can insert the process just before sending the request to the server.
import org.apache.http.HttpRequest;
import org.apache.http.HttpRequestInterceptor;
import org.apache.http.protocol.HttpContext;
public class AmazonizeInterceptor implements HttpRequestInterceptor {
@Override
public void process(HttpRequest request, HttpContext context) throws Exception{
AwsSigner4(request);
}
private void AwsSigner4(HttpRequest request) throws Exception {
/*Implement AWS Signature V4 within this function implementation.*/
}
}
--Modification of HTTP client caller
//...Abbreviation...
void reqFunc(){
//...Abbreviation...
//Executing a request with an HTTP client
HttpClient client = HttpClientBuilder.create()
.addInterceptorLast(new AmazonizeInterceptor()) //← Add this
.build();
CloseableHttpResponse res = (CloseableHttpResponse) client.execute(post);
//...Processing for response...
}
Now, when you make a request with an HTTP client that sandwiches RequestInterceptor, the AwsSigner4 function defined earlier will be executed before sending to the server.
Next, we will implement the AwsSigner4 function that performs the actual signature processing. All you have to do is add "X-Amz-Date" and "Authorization" to the request header, but it takes several steps to find the value of Authorization.
For reference, host, x-amz-date is required for the header information included in the canonicalRequest, but it seems that other header information may be added.
import org.apache.http.util.EntityUtils;
import org.apache.http.HttpEntityEnclosingRequest;
import org.apache.http.HttpRequest;
import org.apache.commons.codec.binary.Hex;
import org.apache.commons.codec.digest.DigestUtils;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.text.SimpleDateFormat;
private void AwsSigner4(HttpRequest request) throws Exception {
/* X-Amz-Date header generation*/
SimpleDateFormat xAmzDateFormatter = new SimpleDateFormat("yyyyMMdd'T'HHmmss'Z'");
xAmzDateFormatter.setTimeZone(Calendar.getInstance(TimeZone.getTimeZone("UTC"))); //Signer expiration date is determined by UTC
String xAmzDate = xAmzDateFormatter.format(new Date()).trim();
request.setHeader("X-Amz-Date", xAmzDate);
/*Authorization header string generation*/
/* 1.Regular request string(canonicalRequest)Generation*/
String path = getPath(request);
String xAmzContentSha = getBodyHash(request);
String canonicalRequest = "POST\n"
+ path + "\n"
+ "\n"
+ "host:" + request.getFirstHeader("Host").getValue() + "\n"
+ "x-amz-date:" + xAmzDate + "\n"
+ "\n"
+ "host;x-amz-date\n"
+ xAmzContentSha;
/* 2.Signature string(StringToSign)Generation*/
String awsRegion = "ap-northeast-1" ; //AWS API request destination region information
String awsNameSpace = "es" ; //Namespace for the AWS service you request
String StringToSign = "AWS4-HMAC-SHA256\n"
+ xAmzDate + "\n"
+ xAmzDate.substring(0, 8) + "/" + awsRegion
+ "/" + awsNameSpace +"/aws4_request\n"
+ DigestUtils.sha256Hex(canonicalRequest);
/* 3.Signature key(SigningKey)Generation*/
String awsSecretAccessKey = "AWS secret access key" ;
// X-Amz-Date → Region → AWS Service Namespace → Fixed Character(aws4_request)Hash in the order of
String hashStr = getHmacSha256ByStrKey("AWS4" + awsSecretAccessKey, xAmzDate.substring(0, 8));
hashStr = getHmacSha256ByHexKey(hashStr, awsRegion);
hashStr = getHmacSha256ByHexKey(hashStr, awsNameSpace);
String SigningKey = getHmacSha256ByHexKey(hashStr, "aws4_request");
/* 4.Authorization header string generation*/
String awsAccessKeyId = "AWS access key ID" ;
String sig = getHmacSha256ByHexKey(SigningKey, StringToSign);
String authorization = "AWS4-HMAC-SHA256 Credential="
+ awsAccessKeyId
+ "/" + xAmzDate.substring(0, 8)
+ "/" + awsRegion
+ "/" + awsNameSpace
+ "/aws4_request,"
+ "SignedHeaders=host;x-amz-date,"
+ "Signature=" + sig;
request.setHeader("Authorization", authorization);
}
/***Get request path*/
private String getPath(HttpRequest req) throws Exception {
String uri = req.getRequestLine().getUri();
//"Separator with URL query string"?Do not include in path
if (uri.endsWith("?")) uri = uri.substring(0, uri.length()-1);
//It seems that there are various types of URL encoding, and it encodes with the logic specified by Amazon.
return awsUriEncode(uri,true);
}
/***Get hash of lix and body*/
private String getBodyHash(HttpRequest req) throws Exception{
HttpEntityEnclosingRequest ereq = (HttpEntityEnclosingRequest) req;
String body = EntityUtils.toString(ereq.getEntity());
return DigestUtils.sha256Hex(body);
}
/***
*AWS specified spec URL encoder
* @param input
* @param encodeSlash
* @return
* @throws UnsupportedEncodingException
*/
private String awsUriEncode(CharSequence input, boolean encodeSlash) throws UnsupportedEncodingException {
StringBuilder result = new StringBuilder();
boolean queryIn = false;
for (int i = 0; i < input.length(); i++) {
char ch = input.charAt(i);
if ((ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9') || ch == '_' || ch == '-' || ch == '~' || ch == '.') {
result.append(ch);
} else if (ch == '/') {
if (queryIn) result.append(encodeSlash ? "%2F" : ch);
else result.append(ch);
} else {
if(!queryIn && ch=='?') {
queryIn = true;
result.append(ch);
}else {
byte[] bytes = new String(new char[] {ch}).getBytes("UTF-8");
result.append("%" + Hex.encodeHexString(bytes,false));
}
}
}
return result.toString();
}
private String getHmacSha256ByStrKey(String strkey, String target) throws Exception {
return getHmacSha256(strkey.getBytes(), target);
}
private String getHmacSha256ByHexKey(String hexkey, String target) throws Exception {
return getHmacSha256(Hex.decodeHex(hexkey), target);
}
/***
*HMAC target string using Key-SHA-Hash to 256
* @param target Hash target
* @param key key
* @return Hex format hash value
*/
private String getHmacSha256(byte[] key, String target) throws Exception {
final Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(key, "HmacSHA256"));
return String.valueOf(Hex.encodeHex(mac.doFinal(target.getBytes()), true));
}
The final summary is as follows. A formatter is generated for each request, and constants are embedded, but please fix it.
import org.apache.http.HttpRequest;
import org.apache.http.HttpRequestInterceptor;
import org.apache.http.HttpEntityEnclosingRequest;
import org.apache.http.protocol.HttpContext;
import org.apache.http.util.EntityUtils;
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.codec.binary.Hex;
import java.text.SimpleDateFormat;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
public class AmazonizeInterceptor implements HttpRequestInterceptor {
@Override
public void process(HttpRequest request, HttpContext context) throws Exception{
AwsSigner4(request);
}
private void AwsSigner4(HttpRequest request) throws Exception {
/* X-Amz-Date header generation*/
SimpleDateFormat xAmzDateFormatter = new SimpleDateFormat("yyyyMMdd'T'HHmmss'Z'");
xAmzDateFormatter.setTimeZone(Calendar.getInstance(TimeZone.getTimeZone("UTC"))); //Signer expiration date is determined by UTC
String xAmzDate = xAmzDateFormatter.format(new Date()).trim();
request.setHeader("X-Amz-Date", xAmzDate);
/*Authorization header string generation*/
/* 1.Regular request string(canonicalRequest)Generation*/
String path = getPath(request);
String xAmzContentSha = getBodyHash(request);
String canonicalRequest = "POST\n"
+ path + "\n"
+ "\n"
+ "host:" + request.getFirstHeader("Host").getValue() + "\n"
+ "x-amz-date:" + xAmzDate + "\n"
+ "\n"
+ "host;x-amz-date\n"
+ xAmzContentSha;
/* 2.Signature string(StringToSign)Generation*/
String awsRegion = "ap-northeast-1" ; //AWS API request destination region information
String awsNameSpace = "es" ; //Namespace for the AWS service you request
String StringToSign = "AWS4-HMAC-SHA256\n"
+ xAmzDate + "\n"
+ xAmzDate.substring(0, 8) + "/" + awsRegion
+ "/" + awsNameSpace +"/aws4_request\n"
+ DigestUtils.sha256Hex(canonicalRequest);
/* 3.Signature key(SigningKey)Generation*/
String awsSecretAccessKey = "AWS secret access key" ;
// X-Amz-Date → Region → AWS Service Namespace → Fixed Character(aws4_request)Hash in the order of
String hashStr = getHmacSha256ByStrKey("AWS4" + awsSecretAccessKey, xAmzDate.substring(0, 8));
hashStr = getHmacSha256ByHexKey(hashStr, awsRegion);
hashStr = getHmacSha256ByHexKey(hashStr, awsNameSpace);
String SigningKey = getHmacSha256ByHexKey(hashStr, "aws4_request");
/* 4.Authorization header string generation*/
String awsAccessKeyId = "AWS access key ID" ;
String sig = getHmacSha256ByHexKey(SigningKey, StringToSign);
String authorization = "AWS4-HMAC-SHA256 Credential="
+ awsAccessKeyId
+ "/" + xAmzDate.substring(0, 8)
+ "/" + awsRegion
+ "/" + awsNameSpace
+ "/aws4_request,"
+ "SignedHeaders=host;x-amz-date,"
+ "Signature=" + sig;
request.setHeader("Authorization", authorization);
}
/***Get request path*/
private String getPath(HttpRequest req) throws Exception {
String uri = req.getRequestLine().getUri();
//"Separator with URL query string"?Do not include in path
if (uri.endsWith("?")) uri = uri.substring(0, uri.length()-1);
//It seems that there are various types of URL encoding, and it encodes with the logic specified by Amazon.
return awsUriEncode(uri,true);
}
/***Get hash of lix and body*/
private String getBodyHash(HttpRequest req) throws Exception{
HttpEntityEnclosingRequest ereq = (HttpEntityEnclosingRequest) req;
String body = EntityUtils.toString(ereq.getEntity());
return DigestUtils.sha256Hex(body);
}
/***
*AWS specified spec URL encoder
* @param input
* @param encodeSlash
* @return
* @throws UnsupportedEncodingException
*/
private String awsUriEncode(CharSequence input, boolean encodeSlash) throws UnsupportedEncodingException {
StringBuilder result = new StringBuilder();
boolean queryIn = false;
for (int i = 0; i < input.length(); i++) {
char ch = input.charAt(i);
if ((ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9') || ch == '_' || ch == '-' || ch == '~' || ch == '.') {
result.append(ch);
} else if (ch == '/') {
if (queryIn) result.append(encodeSlash ? "%2F" : ch);
else result.append(ch);
} else {
if(!queryIn && ch=='?') {
queryIn = true;
result.append(ch);
}else {
byte[] bytes = new String(new char[] {ch}).getBytes("UTF-8");
result.append("%" + Hex.encodeHexString(bytes,false));
}
}
}
return result.toString();
}
private String getHmacSha256ByStrKey(String strkey, String target) throws Exception {
return getHmacSha256(strkey.getBytes(), target);
}
private String getHmacSha256ByHexKey(String hexkey, String target) throws Exception {
return getHmacSha256(Hex.decodeHex(hexkey), target);
}
/***
*HMAC target string using Key-SHA-Hash to 256
* @param target Hash target
* @param key key
* @return Hex format hash value
*/
private String getHmacSha256(byte[] key, String target) throws Exception {
final Mac mac = Mac.getInstance("HmacSHA256");
mac.init(new SecretKeySpec(key, "HmacSHA256"));
return String.valueOf(Hex.encodeHex(mac.doFinal(target.getBytes()), true));
}
}
This time, I actually confirmed the operation only with Elasticsearch API, but if IAM permissions are set correctly, I think that other APIs can be used in the same way. If you have an AWS service that has been confirmed to work, I would appreciate it if you could write it in the comments.
I hope you find it useful.
Recommended Posts