It was when I was asked to look into the environment of a certain development site. The ** environment construction procedure is given verbally **, and Anaconda is installed on Windows and additional packages are installed with pip. Of course, ** version is not specified **, so the latest version at that time will be included, so the minor version will differ depending on the entry timing. In a large team that straddles companies, it is possible that company A works but company B does not. ** Since the introduction of Lint and formatter is optional **, the commit log of git ** the diff of the added and modified part and the formatted part coexist **, and the source review takes time.
Would you like to unify the Python operating environment for all members of the development team? Would you like to create a clean source that is easy to maintain according to the code rules?
Let's start the work style reform of engineers with ** Poetry **!
We will aim for the above. By the way, here are the main problems of the development site mentioned in the overview.
problem | What's wrong | Will Poetry solve it? |
---|---|---|
Oral environment construction procedure(Or txt) | Causes human error such as omission | Reproducibility because the environment installer is completed ◎ |
Pip install on Anaconda environment | In the worst case, you need to reinstall the Anaconda environment. | Don't worry because you don't use Anaconda |
No version specified | It doesn't work depending on the person ... | Version can be unified with one command |
Lint and formatter are optional | Hard-to-read code is hard to maintain | It can be mandatory to install the tool. Autorun needs to be combined with vscode and git. |
We will install poetry under the python environment. It is a one-shot installation with the following command. There is no need to learn operations like Docker.
$pip install poetry
After installing poetry, we will create a template for the project.
$ poetry new xxx
The part corresponding to xxx is the project name. This is the part of * pip install xxx * that corresponds to xxx. You will have a source / test directory and a Readme, pyproject.toml file. After that, push to the git repository and the initial setting is completed.
When executed, it looks like this |
---|
Edit the ** pyproject.toml ** file to configure the dependent libraries required by your project. pyproject.tomlはパッケージビルドに必要なデータを定義するファイルのフォーマットで、setup.py/cfgを置き換えてsetuptoolsではないビルドツールでもビルドが可能となることを目的として作られたものです。
#Add dependent library
$ poetry add xxx
#Added development library
$ poetry add -D xxx
You can run it just like pip install. If you want to specify the version, use the == operator.
$ root@16460604cb93:~/poetry_template# poetry add tensorflow==2.3.0
Creating virtualenv poetry-template-oDQfK0qA-py3.8 in /root/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (48.4s)
Writing lock file
Package operations: 41 installs, 0 updates, 0 removals
$ root@16460604cb93:~/poetry_template# poetry add -D mypy black isort
Using version ^0.790 for mypy
Using version ^20.8b1 for black
Using version ^5.6.4 for isort
As you execute the add command, it will be added to the pyproject.toml file. You can also edit pyproject.toml directly.
[tool.poetry]
name = "poetry_template"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
[tool.poetry.dependencies]
python = "^3.8"
tensorflow = "2.3.0"
[tool.poetry.dev-dependencies]
pytest = "^5.2"
mypy = "^0.790"
black = "^20.8b1"
isort = "^5.6.4"
flake8 = "^3.8.4"
radon = "^4.3.2"
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
We will create a virtual environment (venv) in which the environment according to the conditions described in the pyproject.toml file is installed.
By default, venv is created in the home directory, but it would be nice if it could be created directly under the project, such as when using it with vscode, so change the setting.
$ poetry config virtualenvs.in-project true
$ poetry config --list
Installation is also a command. You will have a .venv directory and a poetry.lock file.
$ poetry install
When executed, it looks like this |
---|
The poetry.lock file is a file that guarantees full dependencies. Even if you didn't specify some explicit versions in the pyproject.toml file, the lock file records version information for dependencies that have been successfully installed in the past so that anyone can reproduce them at any time. In other words, if you ** distribute this file to team members ** and have them ** poetry install **, you will always have a unique environment.
We will access the virtual environment created last. You can access the commands in the virtual environment with the poetry run command.
#Attach to python
$ poetry run python -V
Python 3.8.5
#Test run
$ poetry run pytest
======================================================= test session starts =======================================================
platform linux -- Python 3.8.5, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
rootdir: /root/poetry_template
collected 1 item
tests/test_poetry_template.py . [100%]
======================================================== 1 passed in 0.01s ========================================================
It's been a long time, but that's all for poetry. I hope you understand that you can code the environment with just a few commands, manage it with Git, and all project members can develop in the same environment.
We have created a project template that you can actually use, so please use it. https://github.com/TomokiHirose/poetry_template
I am creating a project template with the following functions. Rewrite the template and use the poetry update command to rewrite the environment to suit the user.
command | Contents |
---|---|
poetry run pytest | ./Run the tests using the file under tests |
poetry run format | black(Powerful formatter)And isort(Arrangement of import order)Runs |
poetry run lint | flake8(linter)And mypy(Static type checking)Runs |
poetry run metrics | Performs static analysis of cyclomatic complexity index, maintenance availability index, LOC, etc. |
poetry run apidoc | Generate API documentation with sphinx |
poetry run testdoc | Generate Test document with sphinx[^1] |
poetry build | ./wheel file under dist(python installer)To generate |
I forgot to install it! Even in that case, instead of "Please install xx", just push the fix to git and all the members hit the update command, and everyone will be in the same environment immediately! Don't you think it's very modern?
Furthermore, in cooperation with vscode, if black and flake8 under .venv are automatically executed at the time of saving, or if black is run before push by linking with the commit hook of git, the format is forced. can do. I don't think it's great to follow PEP8 [^ 2], but if you force ** anyone to write the same code **, even if advanced and newcomers are in the same project It is recommended because it will be easier to maintain because the code like this will be completed.
[^ 1]: Document that records the items to be confirmed in the test and the tester in charge. [^ 2]: PEP8 is a Python coding convention. It is not always necessary to follow it, but there is a difference in ease of maintenance depending on whether there is one standard or not.
Recommended Posts