Switch the package to be installed for each environment with poetry

Note that I'm managing the python environment with Poetry and I want to put different packages for the learning server and the inference server.

background

Basically, I thought it would be good to plunge everything into dependencies, but it seems that there are powerful and vicious packages in the world that require a few GB of memory just by installing, so even poor infrastructure can install it. Hey ... that became a problem.

It sounds reasonable to (temporarily) strengthen your infrastructure, but are you confident that you will definitely resize your instance after the installation is complete over the next few months or years? I don't. So I devised it.

Main subject

poetry has the concepts of ʻoptional and ʻextras. These are options for the ʻadd and ʻinstall subcommands, respectively.

In conclusion, the following steps worked:

1. When adding a dependent package

$ poetry add --optional SOME_STRONG_PACKAGE

2. Add extras package to pyproject.toml with any editor

$ vi pyproject.toml

#↓ Add
[tool.poetry.extras]
train = ["torch", "torchtext"]
↑ Group name ↑ List of packages to install (make sure it is optional in dependencies)

3. Installation (if extras are not needed)

$ poetry install

3'.Installation (if extras required)

$ poetry install -E train

Where it gets stuck

When I try the above method on a project that is already managed by poetry, I get the following error:

[ValueError]        
Extra [train] is not specified.  

After that, I deleted poetry.lock and tried the above poetry install -E train again and it worked. Obviously the error message is unfriendly </ s> PR chance.

Recommended Posts

Switch the package to be installed for each environment with poetry
Switch the module to be loaded for each execution environment in Python
How to set the development environment for each project with VSCode + Python extension + Miniconda
Move what you installed with pip to the conda environment
Create execution environment for each language with boot2docker
How to prepare an environment with different python version and package for each project with pyenv-virtualenv on Amazon Linux
Build a python environment for each directory with pyenv-virtualenv
Display the integrated temperature for each field with Z-GIS
[For beginners] Install the package in the Anaconda environment (Janome)
Switch the setting value of setting.py according to the development environment
Unify the environment of the Python development team starting with Poetry
How to switch the configuration file to be read by Python
Check the operation of Python for .NET in each environment
What to do if the package dependency cannot be repaired
Can't find the package you installed with pip install --user?
Get the package version to register with PyPI from Git
From environment construction to deployment for flask + Heroku with Docker
I want to install a package from requirements.txt with poetry
How to get into the python development environment with Vagrant
Switch PYTHONPATH for each virtualenv
Switch python to 2.7 with alternatives
Switch virtual environment with jupyter
[For IT beginners] What to do when the rev command cannot be used with Git Bash
Become familiar with (want to be) around the pipeline of spaCy
Django cannot be installed in the development environment of pipenv + pyenv
Create an alias for Route53 to CloudFront with the AWS API
Python environment construction 2016 for those who aim to be data scientists
I made a development environment for Django 3.0 with Docker, Docker-compose, Poetry
For the time being, I want to convert files with ffmpeg !!
Jupyter Notebook 6.0.2 cannot be installed in the Python 2.7 environment created in Anaconda
Solution to the problem that Ctrl + z cannot be used in Powershell in Docker for windows environment (provisional)
A memo when creating an environment that can be debugged with Lambda @ Edge for the time being
After getting started, I tried to locally execute httpd conf deployment division for each environment with ansible
Reasons and work notes for the tool / environment kitchen to switch the main PC from Linux to Windows + WSL2
[Python] How to save the installed package and install it in a new environment at once Mac environment
TensorFlow 2.2 can't be installed with Python 3.8!
Prepare the development environment with anyenv
Prepare the environment for Atom for Pythonista
Use Docker Desktop for Windows to start the latest odo with 2 commands
I tried to refer to the fun rock-paper-scissors poi for beginners with Python
[Introduction to Udemy Python3 + Application] 47. Process the dictionary with a for statement
How to easily switch the virtual environment created by Conda on Jupyter
Unable to import packages installed in virtual environment with Anaconda on Windows 10
Use python on Raspberry Pi 3 to light the LED with switch control!
Semi-automatically generate a description of the package to be registered on PyPI
AtCoder writer I wrote a script to aggregate the contests for each writer
Search for Twitter keywords with tweepy and write the results to Excel
Get the number of visits to each page with ReportingAPI + Cloud Functions
I tried with the top 100 PyPI packages> I tried to graph the packages installed on Python
I installed the library with Visual Studio Code, but Unable to import
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_1
Address to the bug that node.surface cannot be obtained with python3 + mecab
A guidebook for doing IoT with MicroPython easily to the last minute
Approach commentary for beginners to be in the top 1.5% (0.83732) of Kaggle Titanic_2
Environment construction to automatically load setup.bash for each workspace moved by ROS