Docker-Client Java API Troubleshooting

Problem Description The previous post is about the basic usage of docker-client Java API https://github.com/spotify/docker-client/. After that, I implemented this API into my code and have been developing some new functions to operate docker containers. However, after several debuggings, I observed there were unexpected volumes in docker volume ls output like below.

root@ubuntu:~# docker volume ls
DRIVER              VOLUME NAME
local               2d399ac2cff4cc75778071dd75ae2f0deb72e744c0f1d9254f25952d19766031
local               6b72bccc07bbb458d009fb8781e39d1a5f3ffdfa4436c80680fc142d21e09943
local               71f34491109d5e5f81c594105f880f5fb3db6a90ca7cbc7e5114e02fbbe02915
local               9c63c935d17a6d7265db01523bc5175ef7cfabc3914f5307a11a7f477c34c855
local               3b526c635946ec523ad46cf547de860c83eea9ecfa874e691ba5eda99778020c
local               019f6049150b572dec743cd07ee3e36c05f528b7727a070179a50ebca406dc1b
local               0331bf7a482c4d3201be4888119ca835a5aec49011d3835488b8b876fa34c707
local               07dfd4547c925f6fc798843642be0173d5bc7c2506455b634b596cd6d4847a65
local               07e102356af18f275cdc0ea1367beb218fc1efc2ce83373b2da1fc080750ff61
local               11d5cfdb10df3302a1d80d8326579921942f843dba95bb4a594f22f6deb5cf12    

Memo Testing docker operations sometimes results in the stack of unused containers and volumes. In that case, you can use the following commands. But be careful when you use the commands as this operation deletes all containers/docker volumes and may cause you data loss.

docker rm -f  $(docker ps -qa) 
docker volume rm $(docker volume ls -qf dangling=true)

Identify the Cause As I had done nothing about the operation for docker volume creatation at the time yet, I doubted this might be because of docker-client Java API mechanizm, or kind of a bug at least. So I read through its documents and source codes but I couldn't find any information mentioning this behavior.

https://github.com/spotify/docker-client/blob/master/README.md https://github.com/spotify/docker-client/blob/master/docs/user_manual.md https://github.com/spotify/docker-client/blob/master/src/main/java/com/spotify/docker/client/DockerClient.java https://github.com/spotify/docker-client/blob/master/src/main/java/com/spotify/docker/client/messages/ContainerCreation.java https://github.com/spotify/docker-client/blob/master/src/main/java/com/spotify/docker/client/messages/HostConfig.java https://github.com/spotify/docker-client/blob/master/src/main/java/com/spotify/docker/client/messages/ContainerConfig.java

After struggling for a while, I found out I totally misunderstood the cause of this issue. When you start a docker container in an usual way such as docker run -it ubuntu:latest, it doesn't create a docker volume aotomatically. So I completely got wrong and thought docker-client API is the one which creates those unintentional volumes. However, when I ran the following docker run command, it returned an unexpected volume while starting a new container.

root@ubuntu:~# docker run -d  rabbitmq rabbitmq:3.6
c29b2b5ac0992a3237af08d36138a65eefa766d59a261d4bc45b5f4a61874f21
root@ubuntu:~# docker volume ls
DRIVER              VOLUME NAME
local               bafe722e39f620712f6a53db824b5a136e987afe17174d33f2ff7ccb52887a5a

Then, I undertood that this is not because of the API but the docker images, RabbitMQ and PostgreSQL, which I am currently using. Without specifying a docker volume to attach, an unexpeced volume is created automatically when starting a new container with the following imges.

Postgres: https://hub.docker.com//postgres/ RabbitMQ: https://hub.docker.com//rabbitmq/

So I tested the following docker container operation. Create a new docker volume, then start the service container with the new volume. It returns no unexpected volume. As a result, now what I am going to do is to implement this procedure into my code.

root@ubuntu:~# docker volume create pgdata
pgdata
root@ubuntu:~# docker run -it -d -v pgdata:/var/lib/postgresql/data postgres:9.6.3
e41a96fde7690a4b9d9639b9f029f43cf5fed808f67bbb58a401d87077cf2f48

root@ubuntu:~# docker ps 
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
e41a96fde769        postgres:9.6.3      "docker-entrypoint..."   6 seconds ago       Up 5 seconds        5432/tcp            adoring_meitner

root@ubuntu:~# docker volume ls
DRIVER              VOLUME NAME
local               pgdata

root@ubuntu:~# docker volume inspect pgdata
[
    {
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/pgdata/_data",
        "Name": "pgdata",
        "Options": {},
        "Scope": "local"
    }
]

Memo What I am wondering is there is no clear explanation about this behavior in the docker image documents. So I am still not sure if it is really the root cause of the issue. I guess, as Postgres and RabbitMQ are stateful service, they are designed to create a new docker volume automatically (if not specified) to store data while starting services. There is still some possibility just I am doing something wrong in the basic docker operation.

Code Modification First of all, you need to create a new docker volume before starting the service container.

        final Volume toCreate = Volume.builder()
                .name(docker_volume_name_for_service_container)
                .driver("local")
                .build();
        docker_client.createVolume(toCreate);

Then, you need to get the mountpoint of the new volume. Mountpoint is necessary in the next process to attach the volume to the service container. If you create a new volume with a name pgdata, its mountpoint should be like /var/lib/docker/volumes/pgdata/_data.

        final Volume volume_info = docker_client.inspectVolume(docker_volume_name_for_service_container);
        final String docker_volume_path = volume_info.mountpoint();

Finally, bind the mountpoint to a certain directory in the service container using HostConfig.builder(). You can find some more explanation about this procedure here https://github.com/spotify/docker-client/blob/master/docs/user_manual.md#mounting-directories-in-a-container.

You can set the local path and remote path in the binds() method on the HostConfig.Builder. Pass binds() a set of strings of the form "local_path:container_path" for read/write.

        final HostConfig hostConfig = HostConfig.builder()
                .appendBinds(String.format("%s:%s", docker_volume_path, path_in_container))
                .build();

Now, you are ready to start the new service container. Pass HostConfig hostConfig to ContainerConfig.builder(). If you need other options for running container, configure other parameters here as well. Then, start the container with docker_client.createContainer()

        final ContainerConfig containerConfig = ContainerConfig.builder()
                .hostConfig(hostConfig)
                .image(service_image)
                .build();
        final ContainerCreation creation = docker_client.createContainer(containerConfig);

Once the above code is implemeted, the unexpected volumes issue should be solved.