Let's write a Qiita article on machine learning for the first time in a while! !! !! I thought, but every time it was troublesome to build the environment, it was a hassle, so it was a meta-like thing that I tried to build an environment for it? I would like to write an article.
In this article, I would like to write about how to easily set up an independent python environment and connect from a browser using Docker-compose. We'll also write about how to execute code interactively from VS Code as well as access from a browser! Even if these are not for writing Qiita, I think that they can answer requests such as "I want to run it in a clean execution environment"!
This article does not cover the introduction of Docker, Docker-compose and VS Code. If you look it up, it will come out easily (Docker on Windows seems to be difficult, but ...). Also, since I specialize in data science, the programming language is Python3. I won't write in detail, but R is also partially supported. Since I am an Apple believer, I use a Mac as the OS, but I think that it will probably work in any environment as long as Docker can be used.
In this article, we will take the following steps:
The docker-stacs repository on jupyter official github has many tools for building a python environment using Jupyter. For example, if you want to do Deep Learning with tensorflow, go to tensorflow-notebook and it will look like below. You can see that there is a Dockerfile in.
In this Dockerfile
ARG BASE_CONTAINER=jupyter/scipy-notebook
There is a part called, but this is said to be an additional operation based on scipy-notebook It makes sense. And if you look at scipy-notebook, it's the same.
ARG BASE_CONTAINER=jupyter/minimal-notebook
It is written that. This is because the environment for tensorflow is the environment for Scipy (for data science with basic python) and tensorflow is added, and the environment for Scipy is the environment where scikit-learn and pandas are added to the minimal environment. It means that there is. I won't go into details, but there are other spark environments and R environments, and if you use the Dockerfile as needed, the environment will be launched in one shot! Also, since it is Docker, if you have a library you want to add personally, you only need to add a little, which is very convenient!
So, first open the directory you want to run from the terminal and clone this repository.
$ git clone https://github.com/jupyter/docker-stacks.git
And, for example, when you want to use base-notebook,
$ cp -R ./docker-stacks/base-notebook ./
If you copy the tool, the download is complete.
It is troublesome to start jupyter every time and write a command that specifies the port, so I will start it easily using docker-compose.
docker-compose.yml
version: '3'
services:
#Service name jupyterlab(Container name)Set to
jupyterlab:
#Build from Dockerfile(To be able to write additional settings)
build:
context: .
dockerfile: Dockerfile
#host's./work in the container/home/Mount on jovyan
# Jovian means "related to Jupiter"
volumes:
- "./work:/home/jovyan"
#Set user
user: root
environment:
B_UID: 1000
NB_GID: 100
GRANT_SUDO: "yes"
#Allow access to the device
privileged: true
#Map host 8888 to container 8888
ports:
- "8888:8888"
#Reboot unless stopped
restart: unless-stopped
#Start jupyterlab after starting the container
command: start.sh jupyter lab --NotebookApp.token=''
It is OK if you create the above file in the same hierarchy as the Dockerfile (I am directly under the base-notebook directory). However, if port 8888 is in use by the host, it is a good idea to set XXXX: 8888 to a different number.
After doing the above, go to the same level as docker-compose.yml in the terminal,
$ docker-compose up -d
Just hit. Then, when it starts up, type localhost: 8888 in your browser![Image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/265221/f45f76e0-4847 -8d03-4309-ddb2a79e5cd6.png) The jupyter lab is opened like this. It's really easy. Since the volume is set to ./work /home/jovyan, if you create a file from the home of jupyterlab, it will be reflected in ./work on the host side. You have now built a python environment. All you have to do is install the required packages according to the article you want to write, and share the Dockerfile for reproducibility!
This is enough, but this time I will challenge to create an interactive execution environment with VS Code as a bonus. The advantage of doing this is that you can still partially execute the py file. I think it's very convenient to be able to easily debug and write processes that are difficult only for ipynb files, such as refactoring.
Then, I will start work immediately. If VS Code with Docker and Remote Desktop Extension installed, press Command + Shift + P to open the command operation tab as shown below. Use Open Folder in Container to open the tool you just set. (In my case, open the base-notebook)
Now, when asked for Config, choose From docker-compose. Then wait for a while, If such a screen is opened, access to the container is successful for the time being.
Speaking of jupyter, type jupyter from the Extension search and install this one displayed at the top.
When the installation is complete, the button "Reload required" will appear as shown below, so press it to reload.
After reloading, let's create a python file by deciding appropriately as jupyter_test.py in ./work. Then type "# %%" as shown below and you'll see commands like "Run Cell" above it! When you press this Run Cell, the code written in the block up to the next "# %%" will be executed interactively. If you can confirm this, you are successful!
This time, with the motivation to write an article on Qiita, I introduced how to easily set up a clean and reproducible python execution environment, and even introduced how to execute it interactively with VS Code. Even if you are not writing a blog, it is convenient to be able to prepare multiple independent python environments with a single command. After all, I realized again that Docker is amazing (vocabulary). In the future, I would like to use this environment to write some articles that are valuable in terms of data science. Thank you for watching until the end! !!
Recommended Posts