Python project environment construction procedure (for windows)

Introduction

This article is for building a development environment for "Python + Pipenv + VS Code".

It's written by an engineer with a year of experience in Python, so you may not be able to understand the culture of Python, but please be aware of that.

motivation

Package management is required when managing source code as a team. Creating the same operating environment reduces communication costs.

Such a natural thing. .. I think that.

Package management is commonplace for me because I've only done app development, but Python isn't limited to app development and is also used for data science and small task automation, so there's a need for package management. I noticed that the perception of is different for each developer.

Also, I think that the fact that the standard package manager does not exist in the Python world (I think) also causes the image of package management to vary from developer to developer.

In the case of javascript, there are npm and yarn, and I think that it is a culture of javascript to start writing code assuming package management. On the other hand, Python has pip as a well-known package manager, but to me who has used npm, pip feels very poor.

I recently introduced pipenv to my team and found it to work, so I'll write how to build a Python project environment using pipenv.

Source of project structure introduced in this article

If you want a base project with pipenv configuration right away, you can find it on github.

PythonProjectStarterKit

If you have already installed Pipenv on global pip, you can immediately try from cloning the above repository to running unit tests. The installation method is as follows.

git clone https://github.com/michiharu/PythonProjectStarterKit.git
cd PythonProjectStarterKit
pipenv install --dev --pre
pipenv run test

About python

Vanilla vs Anaconda

It seems that the standard installer Python is called "Vanilla Python". We will assume that Python will be installed using the Official Page Installer.

Anaconda is convenient, but my opinion is that it doesn't make much sense of the execution environment. If you are an app developer, you need to understand the minimum configuration of the Python execution environment.

It seems good to know Vanilla for application development and Anaconda for data science. The following articles were very helpful for building a Python environment.

Python environment construction various --Windows

When reading this article and rebuilding the Python environment, it is recommended that you output the library you have been using to a text file.

pip freeze > requirememts.txt

Python environment construction

Download standard installer

From the Python official page, select "Files> Windows x86-64 web-based installer" and select Download the installer.

Installation

When you run the installer, there is a checkbox "Do you want to add Path to python in your system environment variables?" Be sure to check before installing.

Verification

Open a command prompt and do the following:

python --version

Environment construction of pipenv

The procedure for using pipenv is explained below.

Installation

You can use it with the following command.

pip install pipenv

When the installation is complete, hit the command pipenv to see a list of commands that can be executed with pipenv.

Setting system environment variables

The default behavior of pipenv is to create a virtual environment resource in .virtualenvs in the USER directory, separate from the project directory. This is inconvenient because it cannot be accessed immediately when checking the Lib of the virtual environment, so enable the setting to create a virtual environment in the project in the system environment variable settings.

PIPENV_VENV_IN_PROJECT=true

The above is the environment construction in preparation for creating a project.

How to create a new project

.vscode settings

Caution

If you set .vscode, VS Code may display the dialog" Do you want to install? ". We do not recommend installing from this dialog.

The reason is that the Pipfile will not be updated because the automatically executed scripts will be installed globally using pip instead of pipenv unless you use the package addition method described below. Install Linter and Formatter with pipenv.

Description of the settings.json file

In order to share Linter and Formatter settings with vscode, the following settings are required in .vscode/settings.json.

{
  "python.venvPath": ".venv",
  "python.pythonPath": ".venv\\Scripts\\python.exe",
  "python.envFile": "${workspaceFolder}/.env",
  "python.linting.enabled": true,
  "python.linting.pylintEnabled": false,
  "python.linting.flake8Enabled": true,
  "python.linting.mypyEnabled": true,
  "python.formatting.provider": "black",
  "python.linting.flake8Args": ["--ignore=E501,W503"],
  "editor.formatOnSave": true
}

The following are possible with the above settings.

About the .env file

In order to add environment variables when executing python, write the following in the .env file. The location is directly under the project.

PYTHONPATH=".\src;"

PYTHONPATH adds a place to look for modules when running python. You can write multiple items separated by semicolons.

About black

Formatter selected black. black himself claims to be "Black is the uncompromising Python code formatter." According to

The settings have been thoroughly discussed about the format, so just use it. Let's stop worrying about the format anymore? It saves time and, more importantly, puts energy into it.

... apparently ...

Therefore, if there is a rule conflict with flake8, black will be given priority. That is, let flake8 ignore the conflicting rules.

python.linting.flake8Args is set to ignore some rules of flake8 to avoid rule conflict with black. For example, --ignore = E501 is a setting to ignore the flake8 rule of up to 79 characters per line (black is up to 88 characters per line). I think there were some other rules that conflicted with black other than the one set as a sample here, but please add it each time you get a flake8 error related to the format.

Introduction of pipenv

Execute the following command. The Python version can be set more finely. If the corresponding version of Python is not installed, an error will occur. In that case, please install from the standard installer.

pipenv --python 3

When you execute the command, the following files will be added to the project directory.

Package added

For packages used at run time

pipenv install [package]

For packages used during development such as Linter and test tools

pipenv install --dev [package]
pipenv install --dev mypy flake8 black isort --pre

--dev is an option to manage as a development package. --pre is an option that allows the installation of pre-released packages. black is described on the official page as "NOTE: This is a beta product", and if there is no pre option, an error will occur indicating that the update of Pipfile.lock failed.

Clone project

If the cloned project has Pipfile, generate .venv with the following command and install the required modules there.

//Pipfile[packages]Add the package described in
pipenv install

//Pipfile[dev-packages]Add the package described in
pipenv install --dev --pre

Enable pipenv in the project managed by requirements.txt

You can install the package from requirements.txt using the argument r.

pipenv install -r ./requirements.txt

Standard directory structure for Python projects

I think it's a culture regardless of language to put the source code in the src directory. The directory structure under src is as follows.

src
├── package
│   ├── foo_module.py
│   ├──   :
│   └── xxx.py
└── tests
    ├── __init__.py
    ├── test_foo_module.py
    ├──   :
    └── test_xxx.py

To import foo_module.py from test_foo_modules.py, write as follows.

import package.foo_module

Execution / script registration

The test can be run with the following command:

//For batch execution
pipenv run python -m unittest discover src -v

If you do not place __init__.py in the tests directory, an error will occur because the test code cannot be found from unittest with the above command.

In addition, frequently used commands can be registered as scripts. Add the following items to the Pipfile.

[scripts]
test = "python -m unittest discover tests -v"
isort = "python -m isort -m 3 ."

With this setting, you can test with the following command.

pipenv run test

The second added isot is a formatter specializing in imports. The black introduced so far does not touch import, so the format of import is left to iceplant.

pipenv run isort

That's all for the procedure for building the environment for a Python project.

Recommended Posts

Python project environment construction procedure (for windows)
python windows environment construction
Python 3.6 installation procedure [for Windows]
Python environment construction (Windows10 + Emacs)
Python environment construction For Mac
Anaconda3 python environment construction procedure
Python3 environment construction (for beginners)
Python environment construction under Windows7 environment
Procedure for building a CDK environment on Windows (Python)
Python environment construction memo on Windows 10
Anaconda python environment construction on Windows 10
[Python3] Development environment construction << Windows edition >>
Python3 TensorFlow for Mac environment construction
Python environment construction
python windows environment
Environment construction (python)
python environment construction
Python --Environment construction
Python environment construction
python environment construction
Python environment construction procedure memo using Docker on Windows10 Home
Python (anaconda) development environment construction procedure (SpringToolsSuites) _2020.4
VScode environment construction (Windows10, Python, C ++, C, Git)
Environment construction procedure: Ubuntu + Apache2 + Python + Pyramid
Windows + gVim + Poetry python development environment construction
Django project environment construction
homebrew python environment construction
Python development environment construction
python2.7 development environment construction
Mac environment construction Python
Python environment construction @ Win7
Install Python (for Windows)
Python environment for projects
VS Code + Azure Functions + Python environment construction procedure
Python environment construction (Anaconda + VSCode) @ Windows10 [January 2020 version]
Build procedure for TensorFlow 2.4.0 (Windows10, CUDA11.1.1, cuDNN 8.0.5, Python 3.8.6)
Image Processing with Python Environment Setup for Windows
Procedure for creating a Python quarantine environment (venv environment)
Prepare Python development environment for each project in Windows environment (VSCode + virtualEnvWrapper + Pylint)
Python + Anaconda + Pycharm environment construction
Environment construction procedure to operate chrome without installing python on Windows (using selenium)
CI environment construction ~ Python edition ~
Build Python environment on Windows
Google App Engine / Python development environment construction procedure (late 2014)
Python environment construction and TensorFlow
Build python environment on windows
[Python] Django environment construction (pyenv + pyenv-virtualenv + Anaconda) for macOS
Procedure to exe python file from Ubunts environment construction
[MEMO] [Development environment construction] Python
[For organizing] Python development environment
[Tensorflow] Tensorflow environment construction on Windows 10
django project development environment construction
Environment construction of python2 & 3 (OSX)
Ansible environment construction For Mac
Install dlib for Python (Windows)
OpenJTalk on Windows10 (Speak Japanese with Python from environment construction)
Environment construction of python and opencv
Get started with Python! ~ ① Environment construction ~
Python + Unity Reinforcement learning environment construction
Install python2.7 on windows 32bit environment
[Django] Memorandum of environment construction procedure