Recently, I've been using Minio, an S3 compatible storage, for automated testing of applications that rely on Amazon S3. Now, create sample code to access Minio from Java (Spring Boot) using the AWS SDK for Java.
It is one of the S3 clones and is compatible with S3, so you can access it with AWS SDK etc. in the same way as Amazon S3. Convenient because you can easily use S3 compatible storage.
$ docker version
docker version
Client:
Version: 1.12.5
API version: 1.24
Go version: go1.6.4
Git commit: 7392c3b
Built: Fri Dec 16 06:14:34 2016
OS/Arch: linux/amd64
Server:
Version: 1.12.5
API version: 1.24
Go version: go1.6.4
Git commit: 7392c3b
Built: Fri Dec 16 06:14:34 2016
OS/Arch: linux/amd64
# java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (IcedTea 3.3.0) (Alpine 8.121.13-r0)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)
# mvn -version
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00)
Maven home: /usr/share/java/maven-3
Java version: 1.8.0_121, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.8-openjdk/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.4.59-boot2docker", arch: "amd64", family: "unix"
We will explain the procedure for operating Minio as a Docker container and accessing it from the Spring Boot application. The created sample code can be found at u6k / sample-minio.
Let's operate it by referring to "Docker Container" of Minio Docs.
$ docker run \
-d \
--name s3 \
-p 9000:9000 \
minio/minio:edge server /export
If it starts normally, the log will be output as shown below.
$ docker logs s3
Created minio configuration file successfully at /root/.minio
Endpoint: http://172.17.0.2:9000 http://127.0.0.1:9000
AccessKey: xxxxxxxxxxxxxxxxxxxx
SecretKey: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Region: us-east-1
SQS ARNs: <none>
Browser Access:
http://172.17.0.2:9000 http://127.0.0.1:9000
Command-line Access: https://docs.minio.io/docs/minio-client-quickstart-guide
$ mc config host add myminio http://172.17.0.2:9000 C05JGTT0UWNXW25Y4IHN zg1JZrFIAWM0dZmXkPF93ZOXXf6Vj3F/QgycmLZE
Object API (Amazon S3 compatible):
Go: https://docs.minio.io/docs/golang-client-quickstart-guide
Java: https://docs.minio.io/docs/java-client-quickstart-guide
Python: https://docs.minio.io/docs/python-client-quickstart-guide
JavaScript: https://docs.minio.io/docs/javascript-client-quickstart-guide
Drive Capacity: 17 GiB Free, 18 GiB Total
Minio starts on port 9000, so connect to port 9000 with a web browser or API. If you try to access http: // localhost: 9000 /, the Minio web interface will be displayed, so try logging in. At this time, please create some buckets. Let's display the list of buckets with the sample code.
The access key and secret key are output to the log, so use this when accessing with a web browser or API. If you want to fix the access key and secret key in automatic test etc., you can set it arbitrarily by specifying the environment variables MINIO_ACCESS_KEY and MINIO_SECRET_KEY.
The procedure for creating a Spring Boot project is omitted. See src / main / java / me / u6k / sample_minio / Main.java for the essential sample code. Some are explained here.
@Value("${s3.url}")
private String s3url;
@Value("${s3.access-id}")
private String s3accessId;
@Value("${s3.secret-key}")
private String s3secretKey;
@Value("${s3.timeout}")
private int s3timeout;
As setting information, S3 endpoint URL, access key, secret key, timeout value (milliseconds) are acquired. Since it is a SpringBoot application, it can be specified in ʻapplication.properties` or an environment variable.
AWSCredentials credentials = new BasicAWSCredentials(this.s3accessId, this.s3secretKey);
Create credentials using the access and secret keys. There seems to be another way to create credentials from a certificate.
ClientConfiguration configuration = new ClientConfiguration();
configuration.setSocketTimeout(this.s3timeout);
configuration.setSignerOverride("AWSS3V4SignerType");
Specify the timeout value and signing method. This area is magical.
AmazonS3 s3 = AmazonS3ClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(credentials))
.withEndpointConfiguration(new EndpointConfiguration(this.s3url, "us-east-1"))
.withClientConfiguration(configuration)
.withPathStyleAccessEnabled(true)
.build();
Initialize the S3 client by setting the credentials, S3 endpoint, client configuration information, and path style.
S3 endpoints specify a region in addition to the URL. I'm hard-coding ʻus-east-1 here, because Minio defaults to ʻus-east-1.
For path styles, you need to specify this because the bucket name in AWS S3 is a DNS name, while Minio is a path.
for (Bucket bucket : s3.listBuckets()) {
L.info("bucket={}", bucket.getName());
}
The list of buckets is acquired and the bucket name is output in the log.
First, build the Docker image.
$ docker build -t sample-minio .
Start the Minio container in advance, create some buckets, and start the sample-minio container.
$ docker run \
--rm \
--link s3:s3 \
-e S3_URL=http://s3:9000/ \
-e S3_ACCESS-ID=xxxxxxxxxxxxxxxxxxxx \
-e S3_SECRET-KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
-e S3_TIMEOUT=10000 \
-v $HOME/.m2:/root/.m2 \
-v $(pwd):/var/my-app \
sample-minio
When completed normally, the log will be output as shown below.
2017-04-30 09:14:54.463 INFO 1 --- [ main] me.u6k.sample_minio.Main : s3.url=http://s3:9000/
2017-04-30 09:14:54.476 INFO 1 --- [ main] me.u6k.sample_minio.Main : s3.access-id=xxxxxxxxxxxxxxxxxxxx
2017-04-30 09:14:54.476 INFO 1 --- [ main] me.u6k.sample_minio.Main : s3.secret-key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2017-04-30 09:14:54.476 INFO 1 --- [ main] me.u6k.sample_minio.Main : s3.timeout=10000
2017-04-30 09:14:57.357 INFO 1 --- [ main] me.u6k.sample_minio.Main : bucket=bar
2017-04-30 09:14:57.357 INFO 1 --- [ main] me.u6k.sample_minio.Main : bucket=boo
2017-04-30 09:14:57.357 INFO 1 --- [ main] me.u6k.sample_minio.Main : bucket=foo
Only the S3 access and the client initialization process are different, but it was confirmed that the others can be accessed in the same way. I think Minio can be used as a local environment for S3.
Recommended Posts