The information I just wanted was not in Japanese, so I will document it. When uploading a file to S3 using the AWS SDK for Java, add some detailed settings.
--Case outline around retry --I will implement it for the time being --Organization of related classes --Other --Reference
I want to upload a file to S3 locally using the AWS SDK for Java. Then, in anticipation of a case where communication is interrupted due to the AWS side or it cannot operate normally due to various reasons, I would like to prepare a retry policy in the AWS client. This time, we will prepare a utility class that returns a singleton of ʻAmazonS3` objects included in the AWS SDK for Java.
In this article, I will focus on "option settings when creating a client", which I was particularly troubled with. Therefore, the description related to the processing before and after the file upload is omitted.
Build a client packed with simple client options. As a flow, we will prepare three, a retry policy instance, a client setting instance, and a client.
First, prepare a method that returns a RetryPolicy instance that contains basic policy settings and AWS-defined strategies.
AmazonS3Utils.java
/**
*Put the appropriate values for baseDelay and maxBackoffTime in the constructor of ExponentialBackoffStrategy.
*Note: Here, an appropriate value is entered as a sample.
*/
private static RetryPolicy getS3BaseRetryPolicy() {
return new RetryPolicy(
new PredefinedRetryPolicies.SDKDefaultRetryCondition(),
new PredefinedBackoffStrategies.ExponentialBackoffStrategy(0, 0),
PredefinedRetryPolicies.DEFAULT_MAX_ERROR_RETRY,
true
);
}
Next, call the method that returns the RetryPolicy instance defined in ↑ to define the client settings.
AmazonS3Utils.java
private static final ClientConfiguration clientConfiguration = new ClientConfiguration()
.withRetryPolicy(getS3BaseRetryPolicy());
Then, initialize the ʻAmazonS3` object, which is the entry point of the S3 client called by other classes and methods, with the client settings.
AmazonS3Utils.java
private static final AmazonS3 amazonS3 = AmazonS3ClientBuilder
.standard()
.withClientConfiguration(clientConfiguration)
.withRegion(Regions.DEFAULT_REGION)
.build();
To summarize the settings so far,
AmazonS3Utils.java
public final class AmazonS3Utils {
/**
*Initialize the client settings to reflect the retry policy.
*/
private static final ClientConfiguration clientConfiguration = new ClientConfiguration()
.withRetryPolicy(getS3BaseRetryPolicy());
/**
*Initialize your Amazon S3 instance.
*/
private static final AmazonS3 amazonS3 = AmazonS3ClientBuilder
.standard()
.withClientConfiguration(clientConfiguration)
.withRegion(Regions.DEFAULT_REGION).build();
/**
*Returns an object that provides a file operation
*/
public static AmazonS3 getS3() {
return amazonS3;
}
/**
*Instantiate a retry policy.
*/
private static RetryPolicy getS3BaseRetryPolicy() {
return new RetryPolicy(
new PredefinedRetryPolicies.SDKDefaultRetryCondition(),
new PredefinedBackoffStrategies.ExponentialBackoffStrategy(0, 0),
PredefinedRetryPolicies.DEFAULT_MAX_ERROR_RETRY,
true
);
}
}
Now you are ready. After that, you can call the S3 client that reflects the settings with the getS3 method.
I would like to organize the roles of multiple classes that appear in the implementation example. I didn't feel like I could do what I wanted to do, maybe because I wasn't sure how to look it up ...
An interface that defines methods for manipulating S3 objects. The concrete implementation seems to be provided by the client builder class. AmazonS3 This object is an interface, and the tool seems to be provided by ʻAmazonS3ClientBuilder` etc.
Originally, before adding the settings around the retry, it was done with ʻAmazonS3ClientBuilder.standard (). Build ();`. I think such a minimalist client will suffice if it requires minimal functionality.
ClientConfiguration is a retry Optional configuration classes that can be added to AWS clients such as proxies and user agent strings.
In this example, I passed RetryPolicy with some customization, but it seems that the default settings can be packed into the client settings by calling the static methodgetDefaultRetryPolicy ()in PredefinedRetryPolicies.
RetryPolicy is A class that builds a retry policy in combination with ClientConfiguration.
Initialize by packing the necessary objects and values in the constructor.
Interfaces such as RetryCondition and BackoffStrategy are specified as arguments in the constructor, but at first I was wondering what to specify for the class that matches the interface of the argument. There are several constructors, but one of them is
RetryPolicy.java
public RetryPolicy(RetryCondition retryCondition,
BackoffStrategy backoffStrategy,
int maxErrorRetry,
boolean honorMaxErrorRetryInClientConfig) {
this(retryCondition, backoffStrategy, maxErrorRetry, honorMaxErrorRetryInClientConfig, false);
}
After a lot of research, I found that the SDK had a class that could be packed with some pre-defined policies.
For the RetryCondition interface, we decided to follow the AWS SDK defaults for this implementation. In PredefinedRetryPolicies There are several types of implementations that can be used, so you can pass them to the RetryPolicy constructor.
If you are not particular about the settings, you can specify the static property DEFAULT_RETRY_CONDITION that returns the default settings.
BackoffStrategy (https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/retry/RetryPolicy.java) interface Extended abstract class [V2CompatibleBackoffStrategy](https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/retry/V2CompatibleBackoffStrategy. java) is provided, and you can set arguments equivalent to the BackoffStrategy interface by using an adapter that extends this abstract class and various concrete classes.
More specifically, [PredefinedBackoffStrategies](https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/retry/PredefinedBackoffStrategies. extended V2CompatibleBackoffStrategyAdapter to java),
Etc. are prepared. When initializing RetryPolicy, it is possible to make detailed settings by somehow passing BackoffStrategy to the constructor.
It's actually an official doc, and I feel like there's something that explains it nicely ... I couldn't find myself, so I would appreciate it if you could point out if there was something like "I'm sorry if I read this even if I can't do that."
-Error retry and exponential backoff on AWS -Retry API call with Exponential Backoff in AWS SDK Java
Recommended Posts