How to use pyenv and pyenv-virtualenv in your own way

Introduction

A colleague said, "The Python development environment is pipenv, isn't it?" And thought, "No, I'm not inconvenienced with pyenv + pyenv-virtualenv." I didn't have anything to pass, so I decided to write it.

What is pyenv / virtualenv

This is for those who are not familiar with pyenv and pyenv-virtualenv.

First, pyenv is "a mechanism that allows you to install multiple versions of python on one machine and switch between them." For example, it would be a hassle to reinstall Python one by one when "new projects use the latest 3.8.5, but old projects in maintenance mode must use 3.6.9". At that time, you can use pyenv to install multiple versions of Python and switch between them.

And virtualenv is "a mechanism to have multiple versions of python environment". This is trying to solve the problem that "python can only register one set of dependent libraries system-wide". This is different from node.js, which creates a directory called node_moduels / for each project and installs dependent libraries under it.

For example, project A depends on requests and pytorch, and project B depends on pandas and numpy. If both projects A and B run on the same version of python, you would have to install both dependent libraries there. But doing so may cause conflicts between libraries. To avoid that, create python 3.8.5 for project A and pythion 3.8.5 for project B. Virtualenv does that for you. pyenv-virtualenv is a mechanism that allows you to use that virtualenv in pyenv in a nice way.

Installation

I think it's easiest for macOS people to use homebrew.

$ brew update
$ brew install pyenv
$ brew install pyenv-virtualenv

For other OS users, or if you really want to use the latest pyenv, you can also clone it directly from the GitHub lipo.

Install pyenv

$ git clone https://github.com/pyenv/pyenv.git ~/.pyenv
$ echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
$ echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
$ echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n  eval "$(pyenv init -)"\nfi' >> ~/.bash_profile
$ source ~/.bash_profile

And install pyenv-virtualenv

$ git clone https://github.com/pyenv/pyenv-virtualenv.git $(pyenv root)/plugins/pyenv-virtualenv
$ echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bash_profile
$ source ~/.bash_profile

If you are using zsh or other shells, please read each configuration file instead of ~ / .bash_profile.

pyenv also has an installer, so you can use it as well. Basically, it just summarizes what you are doing above, but it is easy.

$ curl https://pyenv.run | bash

Basics of usage

Python installation

If you can install pyenv successfully, first install python. Before that, first check the available python version.

$ pyenv install --list

The only unmarked version number is cpython. Besides that, you can choose anaconda, jpython, pypy and so on. Select the version you want to install from them

$ pyenv install <python-version>

If so, that version of python will be installed.

Specify Python to use

You can install multiple versions of Python by repeating the above installation steps. If you want to check the version of Python currently installed

$ pyenv versions

You can see if. For example, if 3.8.5 and 3.7.9 are installed

system
3.7.9
3.8.5

It will be displayed like this. Here, system is not pyenv but python provided by the OS.

So, I specify the Python to be used from this, but there are two types of specification, global and local. Global is a setting that is applied regardless of the directory in which you are located, and local is a setting that is applied under a specific directory. The setting method is as follows.

Global settings

$ pyenv global <python-version>

Local settings

$ pyenv local <python-version>

Then, when starting python, go to the local settings → global settings in that order. For example

$ pyenv global 3.8.5
$ cd /Users/johndoe/projectX
$ pyenv local 3.6.9

If so, python 3.6.9 will be used under / Users / johndoe / projectX, and 3.8.5 will be used in other directories.

Now when I check which version will be used

$ pyenv version

Is used. For example, in the above example

$ cd /Users/johndoe/projectX
$ pyenv version
3.6.9 (set by /Users/johndoe/projectX/.python-version)
$ cd ..
$ pyenv version
3.8.5 (set by /Users/johndoe/.pyenv/version)

It will be. And, as it says "set by ...", the actual version is just written in the file. As you can see from the contents, these are just text files with the version number or written. pyenv looks at it and identifies the version of python to use.

Use pyenv-virtualenv

Next is how to use pyenv-virtualenv. First of all

$ pyenv virtualenv <python-version> <env-name>

Make a copy of a specific version of Python. For example

$ pyenv virtualenv 3.8.5 projectA

Then 3.8.5 for projectA will be created. If you try pyenv version here

system
3.7.9
3.8.5
3.8.5/envs/projectA
projectA

It will be displayed. The environment you just created (copied) is registered as available Python. It looks like you are making two, 3.8.5 / envs / projectA and projectA, but there is only one entity.

The rest is the same as normal pyenv,

$ cd /Users/johndoe/projectA
$ pyenv local projectA

Then you can use projectA as a dedicated 3.8.5 environment. When you don't need it

$ pyenv uninstall projectA

If so, it will be erased cleanly. In this case as well, 3.8.5 itself remains, so if you want to use it in another project, you can create a new environment by using pyenv virtualenv 3.8.5 ....

How to use your own style

When I start a new project, I use it like this.

$ mkdir projectZ
$ cd projectZ
$ pyenv virtualenv 3.8.5 projectZ
$ pyenv local projectZ
$ pip install .... (Required library)
$ pip freeze -l > requirements.txt

In this way, one environment is created for each project. Furthermore, for example, if you want to check the operation with another version,

$ pyenv virtualenv 3.7.8 projecZ-3.7
$ pyenv local projectZ-3.7
$ pyenv install -r requirements.txt

You can switch just by doing.

Summary

I tried to explain pyenv + pyenv-virtualenv that I have been using for many years.

This is a pip issue rather than a pyenv issue, but it has a "weak mechanism for handling dependencies". requirements.txt must be updated manually, and if you create it with pip freeze, it will list not only what you have installed but also the libraries it depends on. When updating the version of the library, individually pip install -U to reflect the changes in requirements.txt. I can't do it for a moment.

In other words, "I'm not inconvenienced by pyenv + pyenv-virtualenv" is not very accurate, and "I'm a little inconvenient, but pyenv + pyenv-virtualenv works in a small turn, so I managed to do it."

Pipenv was created to solve such a problem, but I personally think that poetry seems to be better. And it seems to be quite good to use it in some projects, but in the end I could not throw away pyenv when I put poetry, and I thought that it would be okay as it is.

However, poetry uses venv (python standard virtual environment creation tool) internally to create an environment for each project. So it looks like you can replace pyenv-virtualenv + pip with poetry. I would like to use it a little more aggressively and see if I can do what I have been able to do.

Recommended Posts

How to use pyenv and pyenv-virtualenv in your own way
How to use is and == in Python
How to use Pyenv
How to use pyenv-virtualenv
How to define your own target in Sage
How to use tkinter with python in pyenv
[Introduction to pytorch-lightning] How to use torchvision.transforms and how to freely create your own dataset ♬
How to use Decorator in Django and how to make it
How to install and use Tesseract-OCR
How to use classes in Theano
How to use SQLite in Python
How to create your own Transform
Summary of how to use pyenv-virtualenv
[Introduction to Udemy Python 3 + Application] 36. How to use In and Not
How to create and use static / dynamic libraries in C
How to use .bash_profile and .bashrc
Comparison of how to use higher-order functions in Python 2 and 3
How to install and use Graphviz
How to use Mysql in python
How to use ChemSpider in Python
How to use PubChem in Python
How to use Docker to containerize your application and how to use Docker Compose to run your application in a development environment
Try HeloWorld in your own language (with How to & code)
[Python] When you want to import and use your own package in the upper directory
How to use functions in separate files Perl and Python versions
How to use python put in pyenv on macOS with PyCall
How to display bytes in the same way in Java and Python
How to use calculated columns in CASTable
[Introduction to Python] How to use class in Python?
How to install and use pandas_datareader [Python]
Easy way to use Wikipedia in Python
How to use Anaconda interpreter in PyCharm
How to install your own (root) CA
python: How to use locals () and globals ()
How to use __slots__ in Python class
How to use Python zip and enumerate
How to use regular expressions in Python
How to use Map in Android ViewPager
How to use pandas Timestamp and date_range
Introduction to how to use Pytorch Lightning ~ Until you format your own model and output it to tensorboard ~
How to use the C library in Python
How to use lists, tuples, dictionaries, and sets
Put Linux in your Chromebook and use R ...
Introducing Sinatra-style frameworks and how to use them
How to generate permutations in Python and C ++
How to use Python Image Library in python3 series
Summary of how to use MNIST in Python
[Python] How to use hash function and tuple.
How to use Ruby's PyCall to enable pyenv Python
How to write async and await in Vue.js
How to install Cascade detector and how to use it
How to plot autocorrelation and partial autocorrelation in python
How to pass the path to the library built with pyenv and virtualenv in PyCharm
Tips for those who are wondering how to use is and == in Python
How to use xml.etree.ElementTree
How to use Python-shell
How to use tf.data
How to use virtualenv
Install pyenv and pyenv-virtualenv
How to use Seaboan
How to use image-match