Introducing Kaggle's Docker Image on Windows to build an environment

Record the clogged part as a reminder when you try to easily build an environment using Kaggle's Docker Image in your windows notebook.

Basically, I refer to the article of "Teniro Graffiti", so please refer to this first, except for Win10 Pro whose OS is windows. And if you are using Docker toolbox, it will be helpful.

Stumble point ① Docker-compose yaml execution does not work due to volume name error

When I created the docker-compose yaml file as in the above article, I got an error.

version: "3"
services:
  jupyter:
    build: .
    volumes:
      - $PWD:/tmp/working
    working_dir: /tmp/working
    ports:
      - 8888:8888
    command: jupyter notebook --ip=0.0.0.0 --allow-root --no-browser

image.png I get the error "volume name is too short, names should be at least two alphanumeric characters". Result of investigation, the notation of the environment variable of volumes in the yaml file is $ PWD is a variable, but in the environment at hand It doesn't work, and it seems better to run the $ {PWD} and the PWD (get the current directory path) command. So it worked better with $ {PWD} / tmp / working instead of $ PWD: / tmp / working (because I'm a Docker beginner, I'd appreciate it if you could comment on the reason for more details).

Stumble point ② I can't connect to Docker with a browser

When I ran it with the yaml file modified by docker-compose up --build, I was able to successfully build the container with the jupyter notebook running in the kaggle analysis environment. image.png However, in the browser "http: // localhost: 8888 /? token = XXXX" Attempting to connect to the jupyter notebook via Docker's 8888 port by entering the URL does not work. image.png In the case of Docker toolbox, you can not connect directly to localhost (local host), get the IP address of the virtual machine running with Docker Toolbox and enter it in the URL instead of localhost [Ingenuity](https://qiita.com / jusotech10 / items / b292ac38197926fc6afa) is required.

> docker-machine ip default
192.168.XX.XX

Then I was able to connect to the jupyter notebook successfully. image.png

Stumble point ③ I can't mount the host OS to the directory properly

The volume part of the yaml file mentioned above represents the directory to mount, but in my environment, the mount did not work and the directory on the host OS (local) and the VM did not synchronize. As a result of the investigation, it was found that the cause was as follows.

  1. Oracle VirtualBox folder sharing is not set

  2. The version of the VM boot file (boot2docker.iso) on the docker side does not match the version of VirtualBox that was installed when docker toolbox was installed.

  3. Launched "Oracle VM VirtualBox Manager" and of the docker machine VM you are currently using You can mount it by checking Details> Shared Folder> Add New Shared Folder> Auto Mount / Persist. Since the docker toolbox for windows goes through VirtualBox, the docker command will not be effective unless you make such settings. ** VM VirtualBox icon ** image.png ** VM VirtualBox Manager ** image.png ** VM details shared folder settings window ** image.png However, even if I set about 1., the mount did not work. The cause is that the version of VirtualBox installed first and the version of the boot file of Docker's VM did not match. (I don't know why this happened) I could change the boot file to match the current version of VirtualBox, but I reinstalled VirtualBox to the latest version and updated the boot file at the same time. The boot file is stored as C: //Users/{user}/.docker/machine/cache/boot2docker.iso. This file is available at ** here **. You can find the version of VirtualBox in Help> About VirtualBox, and you can download the installer at ** here **. Replace the boot file, reinstall VirtualBox, and remove the old docker VM machine with VirtualBox Manager. When you start Docker Quick Start Terminal, it will automatically launch a new docker VM based on the boot file. Although it is a small detail, the difference between the boot file version and the VirtualBox version remains the same, and the version of the configuration file called GuestAdditions.iso used to mount the host OS is different, which causes the mount to fail.

Stumble point ④ I don't want to launch jupyter notebook every time

I created a Docker container with a yaml file based on the Docker image, but since the command to start the jupyter notebook is included every time the container is started, the jupyter notebook will start up with the container each time. At that time, by launching the container with the following command and attaching it, you can access the container with the same settings except that you do not launch jupyter notebook in the yaml file settings described above.

docker run -t -i --name {Container name} -p 8888:8888 -v $PWD:/tmp/working:/tmp/working -w /tmp/working {Docker image name} /bin/bash

After creating various installs and directories, jupyter notebook --ip = 0.0.0.0 --allow-root --no-browser You can also launch and use jupyter notebook with . Similarly, you can create and execute the yaml file that matches the docker command above. (I will omit it here) For reference, the docker commands that are often used are listed below.

#Launch the VM
docker-machine start {VM name}
#List of currently created images
docker images
#List of currently created containers
docker ps -a
#Start the container confirmed above by name or ID
docker start {Container name/Container ID}
#Attach to the started container
docker attach {Container name/Container ID}
#Exit the VM
docker-machine stop {VM name}
#Delete unnecessary images
docker rmi {Image name/Image ID}
#Delete unnecessary containers
docker rm {Container name/Container ID}

Finally

If you want to easily analyze with a commercially available windows notebook, but don't want to pollute the environment, Docker is a very useful software. I have summarized the points that I stumbled upon when I introduced it in general Windows 10 Home Edition. I'm glad if you can use it as a reference.

Recommended Posts

Introducing Kaggle's Docker Image on Windows to build an environment
Build Python environment on Windows
Build python environment on windows
How to build a Django (python) environment on docker
Build an environment on windows10 where you can try MXNet
Python 2.7, 3.4, 3.5 extension module build environment on Windows
How to create an NVIDIA Docker environment
Build an LNPP environment on Amazon Linux 2
Building an environment for "Tello_Video" on Windows
Build Linux on a Windows environment. Steps to install Laradock and migrate
Build a LAMP environment on your local Docker
For beginners to build an Anaconda environment. (Memo)
Simply build a Python 3 execution environment on Windows
[Latest] How to build Java environment on Ubuntu
How to build Java environment on Ubuntu (Linux)
Build an Arch Linux environment on Raspberry Pi
Steps to build PyTorch 1.5 for CUDA 10.2 on Windows
How to build an environment for using multiple versions of Python on Mac
Build TensorFlow on Windows
Build XGBoost on Windows
Build a GVim-based Python development environment on Windows 10 (3) GVim8.0 & Python3.6
Build an OpenCV4 environment on Raspberry Pi using Poetry
Build a GVim-based Python development environment on Windows 10 (1) Installation
[Go + Gin] I tried to build a Docker environment
Create an OpenAI Gym environment with bash on Windows 10
Introducing WSL (Ubuntu 18.04) to WIndows10 ~ Proxy environment second part ~
How to build a Python environment on amazon linux 2
Build a machine learning environment natively on Windows 10 (x64)
I tried to build an environment with WSL + Ubuntu + VS Code in a Windows environment
Build an environment for machine learning using Python on MacOSX
How to build a new python virtual environment on Ubuntu
Let's get started with Python ~ Building an environment on Windows 10 ~
Add an extension to build a more comfortable Jupyter environment
Build an Ubuntu python development environment on Google Cloud Platform
How to use VS Code in venv environment on windows
Build a GVim-based Python development environment on Windows 10 (2) Basic settings
Django environment development on Windows 10
Python environment construction procedure memo using Docker on Windows10 Home
Building an environment to execute python programs on AWS EC2
Build GPU environment with GCP and kaggle official image (docker)
Create a Todo app with Django ① Build an environment with Docker
How to manage Python minor version (build virtual environment) on Windows (without Pyenv or WSL)
[Tensorflow] Tensorflow environment construction on Windows 10
How to create an ISO file (CD image) on Linux
Use Xming to launch an Ubuntu GUI application on Windows.
Anyone can understand how to build an initial environment for Python on Mac September 2016 (pyenv + virutalenv)
I tried to create a server environment that runs on Windows 10
[Docker] Build an environment of python (Flask) + GraphQL (graphene) + MySQL (sqlalchemy)
I tried to create an environment of MkDocs on Amazon Linux
The simplest way to build a Spleeter usage environment using Windows
How to build a Python environment using Virtualenv on Ubuntu 18.04 LTS
Build and try an OpenCV & Python environment in minutes using Docker
Building an environment to run ChainerMN on a GPU instance on AWS
Shell script to build pyenv environment on ubuntu in one shot
Everything from building a Python environment to running it on Windows
docker build python based on alpine
Python environment construction memo on Windows 10
Python 3.6 on Windows ... and to Xamarin.
Anaconda python environment construction on Windows 10
Build a go environment using Docker
How to use Dataiku on Windows