Until Docker beginners create Docker images from Dockerfile
Introduction
Since it became necessary to process using Python libraries such as numpy and pandas in an environment not connected to the Internet, until you use Docker to solidify the necessary libraries and files as a Docker image and docker load it on another server I will summarize the procedure of.
We would appreciate it if you could let us know if there are any mistakes in the content or any points that need to be corrected.
environment
- Mac OS Catalina 10.15.6
- Homebrew 2.5.1
Docker preparation
Install and launch
Install Docker from Homebrew by referring to this article, and start Docker.app. How to install Docker on Mac OS with Homebrew --Qiita
I started Docker. I ignored the tutorial, so it's in a clean state. 
By starting Docker, you can type the docker command on the terminal. In addition, Docker.app (GUI screen) will not be used after this, so it is okay to close it.
kaz@Kaz-MBP ~ % docker --version
Docker version 19.03.12, build 48a66213fe
Images and containers
After this, we will proceed on the assumption that we understand the difference between "image" and "container". Regarding "image" and "container", I personally like the explanations on the following sites because they are easy to understand. Difference between docker container and image | Hacknote
Get the original image from Docker Hub
I will do my best to create a Docker image from now on, but apparently it is mainstream to create an original Docker image based on another Docker image instead of creating a Docker image from scratch, so I also do it I will imitate. This time I need Python and its associated libraries, so I'd like to base it on the Docker image that Miniconda3 was on. continuumio/miniconda3 - Docker Hub
Let's pull it now.
kaz@Kaz-MBP ~ % docker pull continuumio/miniconda3
Using default tag: latest
latest: Pulling from continuumio/miniconda3
68ced04f60ab: Pull complete
9c388eb6d33c: Pull complete
96cf53b3a9dd: Pull complete
Digest: sha256:456e3196bf3ffb13fee7c9216db4b18b5e6f4d37090b31df3e0309926e98cfe2
Status: Downloaded newer image for continuumio/miniconda3:latest
docker.io/continuumio/miniconda3:latest
When the pull is complete, you can check the result with the docker images command as shown below. It is OK if a repository named continuumio / miniconda3 is registered. You can see the same result with docker image ls.
kaz@Kaz-MBP ~ % docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
continuumio/miniconda3 latest b4adc22212f1 6 months ago 429MB
Try running the container from the Docker image
Now, from this miniconda3 Docker image, try running the container with the docker run --rm -it continuumio / miniconda3 command.
kaz@Kaz-MBP ~ % docker run --rm -it continuumio/miniconda3
(base) root@8993959bca74:/#
---- rm is an option to automatically delete the container after the container is closed (Docker image is not deleted) --- it is an option for working with containers in an interactive shell --continuumio / miniconda3 is the name of the Docker image, including the slash (not a symbol)
When executed, the prompt display on the left side of the terminal changes. In this state, you can create a container from the Docker image as expected and operate it with an interactive shell.
(base) root@8993959bca74:/# ls
bin dev home lib64 mnt proc run srv tmp var
boot etc lib media opt root sbin sys usr
(base) root@8993959bca74:/#
(base) root@8993959bca74:/# conda info -e
# conda environments:
#
base * /opt/conda
(base) root@8993959bca74:/#
(base) root@8993959bca74:/# python --version
Python 3.7.6
Enter exit to exit the container. Since the --rm option was added when the run command was executed, the container executed at this time disappears.
(base) root@8993959bca74:/# exit
exit
kaz@Kaz-MBP ~ %
kaz@Kaz-MBP ~ % python --version
Python 2.7.16
Now that we have confirmed that this Docker image can be used, let's create an original Docker image based on this.
Creating a Dockerfile
A Dockerfile is like the blueprint you need to create your own Docker image. In the Dockerfile, you need to define the base Docker image and the command you want to execute. This time, while installing numpy, pandas, and jupyter notebook, create a Dockerfile to create a Docker image containing a locally created Python script in the / usr / workspace directory.
FROM continuumio/miniconda3
RUN conda install numpy pandas jupyter -y
RUN mkdir /usr/workspace/
COPY helloworld.py /usr/workspace/
--FROM specifies the original Docker image
--You can also specify a Docker image that has not been pulled
--RUN specifies the shell script you want to execute
--Please note that the execution result of RUN will not be carried over to the next RUN. For example, even if you move the current directory with cd / usr / workspace /, the current directory will be the home directory / in the next RUN.
--COPY places the Python script (helloworld.py) in the same directory as the Dockerfile in the / usr / workspace / directory created by RUN.
Now, create a Docker image from this Dockerfile. After cd to the directory where the Dockerfile is stored, run the docker build -t test . command.
--- t gives a name to the Docker image created this time. This time I named it'test'
--The last. Specifies the directory where the Dockerfile is stored. This time, it is . to specify the current directory.
kaz@Kaz-MBP qiita % docker build -t test .
Sending build context to Docker daemon 3.072kB
Step 1/4 : FROM continuumio/miniconda3
---> b4adc22212f1
Step 2/4 : RUN conda install numpy pandas jupyter -y
---> Running in fce2bbf772f7
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
## Package Plan ##
environment location: /opt/conda
added / updated specs:
- jupyter
- numpy
- pandas
The following packages will be downloaded:
package | build
---------------------------|-----------------
(Omitted)
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Removing intermediate container fce2bbf772f7
---> 1b4aaad5eec1
Step 3/4 : RUN mkdir /usr/workspace/
---> Running in 3c29b1840f3b
Removing intermediate container 3c29b1840f3b
---> 6b9931871d40
Step 4/4 : COPY helloworld.py /usr/workspace/
---> bf07bc8332af
Successfully built bf07bc8332af
Successfully tagged test:latest
kaz@Kaz-MBP qiita %
Since there were 4 lines in total in the Dockerfile, the progress is displayed as Step n / 4.
After creating the Docker image, check the result with the docker images command as before.
kaz@Kaz-MBP qiita % docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
test latest bf07bc8332af 40 seconds ago 2.46GB
continuumio/miniconda3 latest b4adc22212f1 6 months ago 429MB
kaz@Kaz-MBP qiita %
An image named test has been added! Let's run this as a container as well.
kaz@Kaz-MBP qiita % docker run --rm -it test
(base) root@ba61e23ba183:/#
(base) root@ba61e23ba183:/# ipython
Python 3.7.6 (default, Jan 8 2020, 19:59:22)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.18.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import numpy, pandas
In [2]: exit
(base) root@ba61e23ba183:/#
(base) root@ba61e23ba183:/# ls /usr/workspace/
helloworld.py
(base) root@ba61e23ba183:/#
(base) root@ba61e23ba183:/# python /usr/workspace/helloworld.py
Hello, world!
(base) root@ba61e23ba183:/#
(base) root@ba61e23ba183:/#
(base) root@ba61e23ba183:/# exit
exit
kaz@Kaz-MBP qiita %
I was able to run IPython and import both numpy and pandas. /usr/workspace/helloworld.py is also correctly included in the Docker image.
Save Docker image as a file
You can save it with the docker save command. By the way, save seems to be used to save the Docker image, and export to save the Docker container.
Docker save → load export → import which? --Qiita
kaz@Kaz-MBP qiita % docker save test > saved_img.tar
kaz@Kaz-MBP qiita %
kaz@Kaz-MBP qiita % ls
Dockerfile helloworld.py saved_img.tar
kaz@Kaz-MBP qiita %
It is exported as a tar file by docker save. This tar file can be loaded with docker load -i saved_img.tar.
Load Docker image
Finally, with the existing "test" Docker image removed, load the saved saved_img.tar to see if the container works properly.
Delete existing Docker image
kaz@Kaz-MBP qiita % docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
test latest bf07bc8332af 24 minutes ago 2.46GB
continuumio/miniconda3 latest b4adc22212f1 6 months ago 429MB
kaz@Kaz-MBP qiita %
kaz@Kaz-MBP qiita % docker rmi test
Untagged: test:latest
Deleted: sha256:bf07bc8332af060f8711f1610d331c1b5e28555942f1d78b57d8522c04206645
Deleted: sha256:375aa2957526e2da07a6057f47463dfc1568f4eb486df466ecf152c88e1069c4
Deleted: sha256:6b9931871d407a8f3506e64a8e2487a1ef418190308c34cc1d16ea4b97ea310b
Deleted: sha256:33afa4a779be5b75042a62c62563d16940f9d8c71a754317f33d74220f275f4c
Deleted: sha256:1b4aaad5eec1c08bf8b7dd8a3023fdd856c33b835467281d058694e3874d4d9c
Deleted: sha256:77a6450e1f92f5757655aafa4b74ea2c413b7a93b7fbcf1c15741bd3f9c0669e
kaz@Kaz-MBP qiita %
kaz@Kaz-MBP qiita % docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
continuumio/miniconda3 latest b4adc22212f1 6 months ago 429MB
kaz@Kaz-MBP qiita %
Loading a .tar file
kaz@Kaz-MBP qiita % docker load -i saved_img.tar
7ee81e12667e: Loading layer 2.07GB/2.07GB
8f3d70528b79: Loading layer 2.56kB/2.56kB
771c1ba5862d: Loading layer 3.072kB/3.072kB
Loaded image: test:latest
kaz@Kaz-MBP qiita %
kaz@Kaz-MBP qiita %
kaz@Kaz-MBP qiita % docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
test latest bf07bc8332af 28 minutes ago 2.46GB
continuumio/miniconda3 latest b4adc22212f1 6 months ago 429MB
kaz@Kaz-MBP qiita %
Run the container from the loaded Docker image
kaz@Kaz-MBP qiita % docker run --rm -it test
(base) root@d4a072d6c4bd:/#
(base) root@d4a072d6c4bd:/# ipython
Python 3.7.6 (default, Jan 8 2020, 19:59:22)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.18.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import numpy, pandas
In [2]: exit
(base) root@d4a072d6c4bd:/#
(base) root@d4a072d6c4bd:/# python /usr/workspace/helloworld.py
Hello, world!
(base) root@d4a072d6c4bd:/#
(base) root@d4a072d6c4bd:/# exit
exit
kaz@Kaz-MBP qiita %
The library scripts were still available after loading the .tar file! This is what you want to do.
Summary
--In order to prepare a container that runs on Docker, you need to create a Docker image. --This time, I created and built a Dockerfile to create a Docker image. --In the Dockerfile, define the name of the original Docker image and the process you want to perform. --Run the Docker image to run the container
Recommended Posts