Build a comfortable development environment with VSCode x Remote Development x Pipenv

Overview

Manage the Python execution environment with Docker and Pipenv and use it as [VSCode] ](Https://code.visualstudio.com/) Remote Development introduces the mechanism to switch. News that Facebook has partnered with Microsoft for this feature is a hot feature these days. The benefits of introducing this are:

--Docker: Dependent libraries (LAPACK, etc.) and environment variable settings that cannot be managed by Pipenv can also be managed. --Pipenv: Packages required only for development (autopep8, etc.) can be managed separately from the execution environment, and dependency management is easier than pip. --VSCode: Can be seamlessly developed on the docker container built with the above two, and various extensions are powerful

I think that the merit of building a development environment by combining these is great. The image is composed as follows.

image.png

We also introduce recommended extensions and settings for developing Python with VS Code!

Version information

Preparation

--VSCode install: Quickly from Download page --Docker install: Community version is installed and [Non-root running settings](https://docs. docker.com/install/linux/linux-postinstall/)

I think that there are various articles that explain in detail other than the official one, so please check each one if the above is insufficient.

Environmental construction steps

  1. Create a Dockerfile
  2. Create a Pipfile
  3. Describe the build method for the execution environment in docker-compose.yml
  4. Install Remote Development extensions
  5. Describe Remote Development specific settings in docker-compose.extend.yml
  6. Describe the required settings in .devcontainer.json
  7. Type a command with VS Code and build !!

I think that those who operate products using Docker or Pipenv have already made 1 to 3, so please skip to 4. It's a long way to go, but let's do our best to create a comfortable environment!

How to explain

Describe the configuration file with the following structure. I will explain the role of each file and how to create it in order. Replace {repo-name} with the repository name and {tool-name} with the root directory of your development tools.

{repo-name}
├── .devcontainer
│   ├── devcontainer.json
│   └── docker-compose.extend.yml
├── Dockerfile
├── Pipfile
├── docker-compose.yml
└── {tool-name}

Creating a Dockerfile

I'm ashamed to say that I'm not so familiar with Docker, so I think there are many things to do, but for example, create an environment in the following form.

Dockerfile


# Python 3.8 is slim-buster seems to be recommended
#reference: https://pythonspeed.com/articles/base-image-python-docker-images/
FROM python:3.8-slim-buster

#Set environment variables
ENV HOME /root
ENV TZ Asia/Tokyo
WORKDIR $HOME

#Install various libraries used in Python
# software-properties-common and wget are required for later clang 8 install
RUN set -ex \
  && apt-get update \
  && apt-get install -y g++ git openssh-client wget libblas-dev liblapack-dev gnupg software-properties-common make

#clang 8 install
#Since numba depends on llvmlite, install clang 8 series together
RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - \
  && add-apt-repository "deb http://apt.llvm.org/buster/ llvm-toolchain-buster-8 main" \
  && apt-get update \
  && apt-get -y install clang-8 lldb-8 lld-8 gfortran

# llvm-Register the path to config
ENV LLVM_CONFIG=/usr/lib/llvm-8/bin/llvm-config

#host key information known_Described in hosts
#reference: https://gist.github.com/gregdeane/56a7499fddac7251f01dcc9bb64e8486
RUN mkdir -p -m 0600 ~/.ssh \
  && ssh-keyscan github.com >> ~/.ssh/known_hosts

#Pipenv install
RUN pip3 --no-cache-dir install pipenv

#Create a directory with the repository name you want to develop.
RUN mkdir {repo-name}

Since it is assumed that we will develop a private repository on GitHub this time, we will not clone the repository or install the dependencies in the Dockerfile. Docker-compose side does not yet support --ssh default, so install it by another method. If you develop in public repository, you should also do git clone and pipenv install --dev in Dockerfile.

Creating a Pipfile

While adding the necessary libraries to the execution environment, list the tools required for the development environment under [dev-packages]. For example

Pipfile


[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
Cython = "*"
joblib = "*"
numpy = "*"
scipy = "*"
scikit-learn = "*"
lightgbm = "*"

[dev-packages]
nose = "*"
coverage ="*"
flake8 = "*"
isort = "*"
Sphinx = "*"
sphinx-autodoc-typehints = "*"
autopep8 = "*"
setuptools = "*"
mypy = "*"

[requires]
python_version = "3.8"

It is like this. First of all, let's actually do docker run based on the above Dockerfile and check if this Pipfile works properly.

Introduction of Remote Development

Install based on Turorial of Remote Development. It can be easily installed from the VS Code GUI below. image.png

It's easy.

Described how to build for operating environment in docker-compose.yml

Create a docker-compose.yml that builds the above Dockerfile. For Dockerfiles that require you to specify build-arg, set it here. The simplest example is

docker-compose.yml


version: "3.7"
services:
  dev:
    build:
      context: .
      dockerfile: Dockerfile

Is it a shape like?

Remote Development specific settings are described in docker-compose.extend.yml

It's the production from here. Create a docker-compose.extend.yml that consists only of the settings required for Remote Development. By passing two yaml files in .devcontainer.json created in the next section, the build will be done with the settings to which each setting is applied in order. For example

yaml:docker-compose.extend.yml


version: "3.7"
services:
  dev:
    volumes:
      - ~/.ssh/id_rsa:/root/.ssh/id_rsa:ro
    command: sleep infinity

Make it like this.

In the item volumes, the local public key is mounted on the Docker image used in Remote Development. Therefore, you need to register the public key of your local machine on GitHub. Please refer to here for details. Again, this is an unnecessary item because authentication is not required when developing with a public repository.

Also, command is mandatory this time. Without this, Remote Development would just build and die ...

Describe the required settings in .devcontainer.json

It's another breath when you come here. It is a file that specifies how to build the Dockerfile created above. Below is an example and the role of each parameter in the comments.

json:.devcontainer.json


{
    #Become the name of Workspace.
	"name": "{repo-name} DevEnv",

    #Root directory used to specify the location of the configuration file.
	"context": ".",

    # docker-List of paths to yml used in compose up.Strict order.
	"dockerComposeFile": [
		"../docker-compose.yml",
		"docker-compose.extend.yml"
	],

    #Docker with build settings for development environment-compse.service name on yml.
	"service": "dev",
	"shutdownAction": "stopCompose",

    #Root directory to be a VS Code Project.
	"workspaceFolder": "/root/{repo-name}",

    #Command to execute when Docker build is completed.Clone and install the repository here.
	"postCreateCommand": "git clone -b {branch name} {GitHub url} . && pipenv install --dev",

    #Settings to apply to the new VS Code launched in Remote Development.
	"settings": {
		"autoDocstring.docstringFormat": "sphinx",
		"autoDocstring.guessTypes": false,
		"editor.formatOnSave": true,
		"editor.suggestSelection": "first",
		"git.autofetch": true,
		"git.confirmSync": false,
		"kite.showWelcomeNotificationOnStartup": false,
		"python.formatting.autopep8Args": [
			"--max-line-length=200"
		],
		"python.linting.pylintEnabled": false,
		"python.linting.flake8Args": [
			"--max-line-length=200"
		],
		"python.linting.flake8Enabled": true,
		"python.linting.mypyEnabled": true,
		"python.jediEnabled": false,
		"terminal.integrated.inheritEnv": false,
		"vsintellicode.modify.editor.suggestSelection": "automaticallyOverrodeDefaultValue"
	},

    #If you register the ID of the extension you want to use in the Remote environment, it will be installed automatically when you build the environment..
	"extensions": [
		"ms-python.python",
		"njpwerner.autodocstring",
		"ms-azuretools.vscode-docker",
		"visualstudioexptteam.vscodeintellicode"
	]
}

To supplement the extension,

njpwerner.autodocstring A tool that automatically generates docstrings for functions and classes in Python. I want to automatically generate html for documents with Sphinx, so " autoDocstring .docstringFormat ":" sphinx " also sets the style of the docstring.

visualstudioexptteam.vscodeintellicode A super useful tool for Python that does very powerful completion etc. " python.jediEnabled ": false is a required setting.

python.linting I think it's a sin not to mention Type Hinting in Python3 series, so "python.linting.mypyEnabled ": true is mandatory, and it's hard to read that max-line-length is the default on current monitors. Since there is no help for it, " --max-line-length = 200 " is specified for each of ʻautopep8 and flake8`. The default linter in Python for VS Code is pylint, but when I used it, I had a lot of addiction around import, so [flake8](http:: //flake8.pycqa.org/en/latest/) is used.

It's like that.

Type a command with VS Code and build !!

(For Ubuntu) Open the command window with Ctrl + Shift + p on VS Code and hit ʻOpen Folder in Container ...` as shown below.

image.png

If you specify the directory where the local Dockerfile is located, build will start automatically and VS Code that runs on the environment built on Docker Container will be newly launched. At the first time, it takes time to build docker and pipenv install, so please wait patiently.

Finally, Python's Interpreter may not recognize the pipenv environment, but running reload window from the command window will fix it.

This completes the development environment! Thank you for your hard work! !!

Recommended Posts

Build a comfortable development environment with VSCode x Remote Development x Pipenv
Build a Go development environment with VS Code's Remote Containers
Build a C language development environment with a container
[Python] Build a Django development environment with Docker
Build a Django development environment with Doker Toolbox
Build a machine learning application development environment with Python
Build a development environment with Poetry Django Docker Pycharm
Ssh to virtual environment with remote development of vscode
Build a Django development environment with Docker! (Docker-compose / Django / postgreSQL / nginx)
[Django] Build a Django container (Docker) development environment quickly with PyCharm
How to build a python2.7 series development environment with Vagrant
Create a simple Python development environment with VSCode & Docker Desktop
Build a python environment with pyenv (OS X El Capitan 10.11.3)
Create a Python execution environment for Windows with VScode + Remote WSL
Build a comfortable psychological experiment / analysis environment with PsychoPy + Jupyter Notebook
[DynamoDB] [Docker] Build a development environment for DynamoDB and Django with docker-compose
Build a Tensorflow environment with Raspberry Pi [2020]
Build a Fast API environment with docker-compose
[Linux] Build a jenkins environment with Docker
Build a python virtual environment with pyenv
Build a modern Python environment with Neovim
[Linux] Build a Docker environment with Amazon Linux 2
Comfortable Jupyter Lab (Python) analysis environment created with Docker + VSCode + Remote Container
I tried to build a Mac Python development environment with pythonz + direnv
Build a WardPress environment on AWS with pulumi
Build Django + NGINX + PostgreSQL development environment with Docker
Build the fastest Django development environment with docker-compose
Build Python development environment with Visual Studio Code
Go (Echo) Go Modules × Build development environment with Docker
Build a python environment with ansible on centos6
Start Django in a virtual environment with Pipenv
Create a python3 build environment with Sublime Text3
Build a Django environment with Vagrant in 5 minutes
Build a Python development environment on your Mac
Build a virtual environment with pyenv and venv
Build a Kubernetes environment for development on Ubuntu
Build a Python environment with OSX El capitan
Build a Minecraft plugin development environment in Eclipse
Quickly build a Python Django environment with IntelliJ
Build a mruby development environment for ESP32 (Linux)
Build a Python machine learning environment with a container
Build a Python development environment on Raspberry Pi
Build a python execution environment with VS Code
Get a quick Python development environment with Poetry
Cross-compiling Raspberry Pi and building a remote debugging development environment with VS Code
Build a TensorFlow development environment on Amazon EC2 with command copy and paste
Build a local development environment with WSL + Docker Desktop for Windows + docker-lambda + Python
Build a GVim-based Python development environment on Windows 10 (3) GVim8.0 & Python3.6
Build a python virtual environment with virtualenv and virtualenvwrapper
Build a local development environment for Laravel6.X on Mac
Repairing a broken development environment with mavericks migration (Note)
Build a python environment for each directory with pyenv-virtualenv
Create a comfortable Python 3 (Anaconda) development environment on windows
Create a python development environment with vagrant + ansible + fabric
Build a GVim-based Python development environment on Windows 10 (1) Installation
Build a python virtual environment with virtualenv and virtualenvwrapper
QGIS3 Python plugin development environment construction with VSCode (macOS)
How to build a development environment for TensorFlow (1.0.0) (Mac)
Build a Python development environment using pyenv on MacOS
Python development environment with Windows + Python + PipEnv + Visual Studio Code
Create a Python development environment on OS X Lion