I tried using Pipenv

Introduction

It's been about half a year since I first learned about Pipenv and started using it. I will summarize what I thought was convenient.

To put it simply, Pipenv is convenient, so everyone should try it.

So far

Until I learned about the existence of Pipenv, I used venv to create a virtual environment and managed the modules listed in requirements.txt by installing them with pip.

requirements.txt


requests==2.22.0

Prepare such a file

$ python3 -m venv venv
$ ./venv/bin/python -m pip install -r requirements.txt
Collecting requests==2.22.0 (from -r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5 (from requests==2.22.0->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests==2.22.0->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests==2.22.0->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests==2.22.0->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Installing collected packages: idna, certifi, urllib3, chardet, requests
Successfully installed certifi-2019.11.28 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.7

Something like this.

If you want to execute code in this environment, you can either source ./venv/bin/activate or specify ./venv/bin/python directly.

Problems so far

There were some problems with the management of modules so far.

It's hard to see which modules depend on

requirements.txt


requests==2.22.0

At first, I was worried that the version of the dependent module could not be fixed with this way of writing requirements.txt. However, the pip command has a subcommand called freeze, which outputs all currently installed modules in a format that can be described in requirements.txt.

As a result, my requirements.txt looks like this.

requirements.txt


certifi==2019.11.28
chardet==3.0.4
idna==2.8
pkg-resources==0.0.0
requests==2.22.0
urllib3==1.25.7

Now you can specify the version of all dependent modules! But the big problem here is

** I don't know the module dependencies at all **

As I will explain later, in this respect I have become a body that Pipenv cannot part with.

The current requirements.txt just lists all the modules that will be installed, so I don't see any dependencies on each. Also, depending on the project, the files are separated into dev-requirements.txt and build-requirements.txt (as was the case with the project I am developing). I had a hard time trying to give one version of the module. I don't know which package it depends on, so I installed it in another environment and checked the dependent packages.

It is troublesome to execute the code

Until I created the virtual environment and ran the code, I had the following steps:

$ python3 -m venv venv  #Create virtual environment
$ ./venv/bin/pip install -U setuptools pip wheel  #Update the basic module to the latest version
$ ./venv/bin/pip install -r requirements.txt  #Installation of required modules
$ ./venv/bin/python main.py  #Code execution in a virtual environment

What! It would be nice to keep the venv environment once created, but that is not the case.

Don't worry if you use Makefile

I use Makefile to make this possible with a single command.

Makefile


venv:
  python3 -m venv venv
  ./venv/bin/pip install -U setuptools pip wheel
  ./venv/bin/pip install -r requirements.txt

.PHONY: run
run:
  ${MAKE} venv
  ./venv/bin/python main.py

Now you can run the code with make run at any time.

I got to know Pipenv

One day I met Pipenv. .. ..

https://pipenv-ja.readthedocs.io/ja/translate-ja/

The official documentation for Pipenv is very good. It's Japanese and easy to read. Personally, this was a great help.

Good points of Pipenv

You can see the function of Pipenv by looking at the official documentation. Here, try using Pipenv and use the management method so far. Here are some of the features that I found useful.

Easy to understand module dependencies

It is no exaggeration to say that I wrote this article because I wanted to say this.

I won't go into detail here, but Pipenv manages modules in a file called Pipfile instead of requirements.txt. This file is automatically generated when you run the pipenv command, so you may not have a chance to edit it directly.

Pipfile


[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
requests = "==2.22.0"

[requires]
python_version = "3.6"

Now try running the command pipenv graph.

$ pipenv graph
requests==2.22.0
  - certifi [required: >=2017.4.17, installed: 2019.11.28]
  - chardet [required: >=3.0.2,<3.1.0, installed: 3.0.4]
  - idna [required: >=2.5,<2.9, installed: 2.8]
  - urllib3 [required: >=1.21.1,<1.26,!=1.25.1,!=1.25.0, installed: 1.25.7]

** Not only can you check the installed modules, but you can also see the dependencies of each. ** ** This was personally most appreciated.

Since you can see the version required for each module, you do not have to check the dependencies each time you want to change the version.

Easy to execute code

If you are using Pipenv, you can create a virtual environment → install necessary modules → execute code in the virtual environment with the following command.

$ pipenv install  #Creating a virtual environment+Installation of required modules
$ pipenv run python main.py   #Main in virtual environment.Run py

The number of commands is clearly reduced compared to the procedure introduced in [Troublesome to execute code](#Troublesome to execute code). With this amount, you don't have to bother to use Makefile.

scripts can be defined

By defining a section called scripts in Pipfile, you can define frequently executed statements as commands. Click here for details (https://pipenv-ja.readthedocs.io/ja/translate-ja/advanced.html#custom-script-shortcuts) (Official)

For example, the Pipfile that defines the command to execute the test is as follows.

Pipfile


[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]
pytest = "==5.3.1"

[packages]
requests = "==2.22.0"

[requires]
python_version = "3.6"

[scripts]
test = "python -m pytest tests/"

A section called [scripts] has been added.

It allows you to define a subcommand that follows the pipenv run command, in which case the pipenv run test will run the test with the pytest. Frequently used commands such as test execution and packaging were described in Makefile, but if you use this, you only need one Pipfile to execute simple code.

Reads .env file

When you run the code with pipenv run, it will automatically load the .env file. Click here for details (https://pipenv-ja.readthedocs.io/ja/translate-ja/advanced.html#automatic-loading-of-env) (Official)

This is also sober.

I use this function like a configuration file. Write the value in the .env file in the format key = value so that the code reads the value from the environment variable.

.env


KEY=VALUE

main.py


import os
key = os.environ.get("KEY")
print(key)  # VALUE

Just like the configuration file, just editing the file in .env will change the execution result.

$ pipenv run python main.py
VALUE

If the scale of the code becomes large, it is better to create a mechanism to read the configuration file properly, This function is very helpful because it is troublesome to write the code to read the configuration file by using ConfigParser etc. for small code.

Try using Pipenv

If you are using Python and have never heard of Pipenv, please try it.

Recommended Posts

I tried using Pipenv
I tried using argparse
I tried using anytree
I tried using aiomysql
I tried using Summpy
I tried using coturn
I tried using matplotlib
I tried using "Anvil".
I tried using Hubot
I tried using ESPCN
I tried using openpyxl
I tried using Ipython
I tried using PyCaret
I tried using cron
I tried using ngrok
I tried using face_recognition
I tried using Jupyter
I tried using PyCaret
I tried using Heapq
I tried using doctest
I tried using folium
I tried using jinja2
I tried using folium
I tried using time-window
I tried using pipenv, so a memo
[I tried using Pythonista 3] Introduction
I tried using easydict (memo).
I tried face recognition using Face ++
I tried using Random Forest
I tried using BigQuery ML
I tried using Amazon Glacier
I tried using git inspector
[Python] I tried using OpenPose
I tried using magenta / TensorFlow
I tried using AWS Chalice
I tried using Slack emojinator
I tried using Rotrics Dex Arm # 2
I tried using Rotrics Dex Arm
I tried using GrabCut of OpenCV
I tried using Thonny (Python / IDE)
I tried server-client communication using tmux
I tried reinforcement learning using PyBrain
I tried deep learning using Theano
Somehow I tried using jupyter notebook
[Kaggle] I tried undersampling using imbalanced-learn
I tried shooting Kamehameha using OpenPose
I tried using the checkio API
[Python] I tried using YOLO v3
I tried asynchronous processing using asyncio
I tried PyQ
I tried AutoKeras
I tried papermill
I tried django-slack
I tried Django
I tried spleeter
I tried cgo
I tried using Amazon SQS with django-celery
I tried using Azure Speech to Text.
I tried using Twitter api and Line api
I tried playing a ○ ✕ game using TensorFlow
I tried using YOUTUBE Data API V3