[JAVA] I was addicted to not being able to connect to AWS-S3 from the Docker container

I was unexpectedly addicted to trying to get the files on AWS-S3 from the application (JVM) in the Docker container launched on the AWS-EC2 instance. I'll note that this information may be useful to someone. If you are in a hurry, please see only the final summary.

Cause

The error I was getting was something like this.

Unable to load AWS credentials from any provider in the chain
        at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:131)
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3820)
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3778)
        at com.amazonaws.services.s3.AmazonS3Client.listObjectsV2(AmazonS3Client.java:649)

I was angry without the credential.

What I tried part 1

If this did not pass, I could not even start the application, so for the time being, I set the environment variables as follows in the container and tried restarting.

export AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxx
export AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxx

As a result, I got the same error. It doesn't seem to read the environment variables. ʻEnv | grep AWS` Then, it seems that the environment variables are set properly.

What I tried part 2

Next, for the time being, I created a credential file in the container. As the application launch user, I created the following file with the name $ HOME / .aws / credentials.properties.

accessKey=AKIAJZJ4UAXAC4QVVW5Q=xxxxxxxxxxxxxxxx
secretKey=FXNnRbZjAlg5Z9SWkFbpKXKLRpGVKLTlwNoKDHuP=xxxxxxxxxxxxxxxx

As a result, I got the same error. It doesn't seem to read credentials.properties. I also prepared credentials just in case, but it didn't work.

What I tried part 3

Finally, when I came here, I decided to set it properly when creating the container. I remember the word hurry.

I modified docker-compose.yml to look like the following.

version: '2.0'
services:
  my-app:
    command: /bin/bash
    container_name: my-app
    image: xxxx/xxxx/my-app:my-app-1.2.3
    network_mode: bridge
    ports:
      - 9000:9000/tcp
    stdin_open: true
    tty: true
    volumes:
      - /var/log/my-app:/var/log/my-app:rw
environment:                              #← Add
  - AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxx    #← Add
  - AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxx #← Add

As a result, I got the same error. It doesn't seem to read the environment variables. I also tried creating a ʻenv file, but it didn't work. When I do ʻenv | grep AWS in the container, it looks like the environment variables are set properly.

What I tried part 4

At this point, I thought, "It doesn't matter because I'm angry that there is no credential, "but can I connect to the S3 bucket I want to see from this EC2 instance in the first place? I started to worry about that. There is no help for it, so install ʻaws-clion the host machine and check the connection. Create thecredentials.properties` file mentioned above on the host machine.

# credentials.Properties creation
touch $HOME/.aws/credentials.properties
(Write and save the access key and secret key with vi)

#Introduced because pip was not included
curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
sudo python get-pip.py

# aws-cli introduction
sudo pip install awscli

#Verification
aws s3 ls s3://xxxxx/xxxxx

I was able to get the file list from S3 safely. It seems that the host machine can connect without any problem.

What I tried part 5

Try mounting credentials.properties on the host machine when creating the container. I rewrote the docker-compose.yml file as follows. The directory on the container side should be the .aws directory under the home directory of the application startup user.

version: '2.0'
services:
  my-app:
    command: /bin/bash
    container_name: my-app
    image: xxxx/xxxx/my-app:my-app-1.2.3
    network_mode: bridge
    ports:
      - 9000:9000/tcp
    stdin_open: true
    tty: true
    volumes:
      - /var/log/my-app:/var/log/my-app:rw
      - $HOME/.aws/credentials.properties:/home/my-app/.aws/credentials.properties #← Add

As a result, I was able to successfully retrieve the file from S3 and launch the application! Why didn't the other method work? Has not been properly investigated and is unknown.

Summary

Below is a summary of my personal situation

--Premise: You can connect to S3 from the host machine --Why it didn't work in other ways → I don't know (not investigated)

I'm sorry for the dragonfly that is cut off, but that's all from the scene. If you have any idea of the cause, I would appreciate it if you could tell me.