UpNext2 Development Record # 1 Build Python CI environment in VS Code

This is a continuation of [UpNext2 Development Record # 0] 1. This time, when creating preprocessing in Python, we will set up an environment that satisfies the goals C1 to C3 presented last time. The local environment is MacOS Catalina.

Goals C1 to C3 are reposted.

Item item Remarks
C1 Version management with GitHub Previously, version management was not performed. This time, we will use GitHub to record and publish the development process. The GitHub release of V1 was just a file copy after the development was completed.
C2 Using VS Code Conventionally Flutter/Made only with Dart and uses Android Studio. This time I plan to use Python for preprocessing, so it's a good opportunity to develop with VS Code.
C3 Test description Since it is a personal development, I have not written a test. However, in UpNext, when I stepped on a complicated bug in the development of a certain scale, I was keenly aware of the need for testing. I decided to write a test properly for studying.

The following is a step-by-step overview of the setup.

1. Preparation of GitHub and cooperation with local repository handled by VS Code

After trying various things, the following procedure was the best for successfully linking local and remote.

This makes it possible to perform staging to commit to the local repository and push to the remote repository by operating VS Code on the UI. (It may be necessary to set the user name etc. with the git command)

In addition, it is fun to include GitGraph etc. as an add-on for VS Code.

2. Set files not to be saved in the repository with .gitignore

Do not push work directories and files of various tools, work for development, and information that you do not want to publish (API key obtained by yourself, etc.) to the repository. The mechanism for that is .gitignore.

2.1. Creating a .gitignore file and defining its contents

Create a .gitignore file in the root directory of your project. The contents are as follows for now.

.gitignore


.*
*cache*
*local*
*secret*

In particular,. * And \ * cache * will be required. Without it, a large number of system work files and common libraries for python virtual environments would end up in the repository.

2.2. Don't push .gitignore itself

I like to do this, but it is possible to avoid pushing .gitignore itself.

Add excludesfile = .gitignore in the [core] group of .git / config in your project home.

3. Build a Python virtual environment with venv

In VScode, you can select and use the Python environment, but when preparing various things, let's make sure that Python 3 is used by default. Adjust the path so that 3 series appears as python -V. Make sure that pip is also 3 series. (Note that recent Python doesn't recommend typing pip directly, it seems to use it as python -m pip)

Since Python is complicated depending on the version of the library, it is recommended to prepare a virtual environment for each application and switch the library set. Previously, pyenv and virtualenv were used to switch to include the python version itself, but they are not recommended in Python 3.6 and later. It is now recommended to use venv.

In your project home, set up your venv environment as python -m venv .pyvenv. .pyvenv can be any other name, but it's best to start with. So that it's subject to .gitignore. Various common libraries are installed in the venv environment, so if you do not hit .gitignore, it will be a big deal. If you create a new venv environment, the library including pip will be in the initial state, so please re-install it according to various instructions.

After setting up the venv environment, you will be able to select the venv environment on VS Code, so please select it (at first, it does not appear even if you click the status bar, select it from Python: Select Interpreter in the command palette menu. Seems to need to). If you select it here, VS Code will automatically activate venv. Of course, you'll need to install VS Code's Python extension in the meantime.

4. Build a pytest environment

Currently, it seems that the library built-in pytest is often used instead of the Python built-in unittest. Select pytest from Python: Configure Tests in the command palette menu to build the environment. At this time, select .Root directory as the test directory.

To keep your sources and tests clean, divide your directories as follows: In addition, since we are planning to code other than Python in this plan, it is actually divided into subprojects.

(Project root)/
 ├ src - temp.py
 └ tests - test_temp.py

Here, if you import temp.py from test_temp.py with a normal relative path specification, you will get addicted to an error.

By applying the updated solution of "[I wrestled with ModuleNotFoundError that occurs when I put the test code and the code to be tested in separate directories in Python] 2", I was able to solve it elegantly. It was.

The solution is:

By combining the above with the test directory as .Root directory in the pytest settings, you will be able to execute the test normally (Note that if you execute test_temp.py directly without using the VSCode UI, the import will fail. ). In addition, the correspondence based on VS Code is also posted in the reference link, this time it was done by trial and error.

5. Setting coding standards

In the command palette menu, under Python: Select Linter, select flake8. There may be various other settings, but I don't remember for a moment. Sweat

6. Set up automated test when committing local repository as CI

The coding standard check can be automatically checked at save time with VS Code, but the test must be executed manually. Originally, I want the test to be executed automatically when committing the local repository and when pushing to the remote repository. The latter is useful when the committer is different from the pull requester, and it seems that Travis CI used to be popular and GitHub Actions is used these days.

For personal development, it seems that only automatic testing at the time of committing the local repository is enough, so let's set it. It seems to be a good idea to use the hook function of the local git command to execute a test with the hook script of pre-commit. Create the project home /.git/hooks/pre-commit with execute permission as follows. Note that this script is outside the jurisdiction of VS Code, so don't forget to write activate.

pre-commit


#!/bin/sh
source .pyvenv/bin/activate
python -m pytest

Now when you commit to a local repository in VS Code, the test will run automatically and the commit will be canceled if the test fails. It's a success.

Recommended Posts

UpNext2 Development Record # 1 Build Python CI environment in VS Code
Development environment in Python
Build Python development environment with Visual Studio Code
Use Python in Anaconda environment with VS Code
Build a python execution environment with VS Code
[ev3dev × Python] Build ev3dev development environment
Allow real-time code checking in Python development with VS Code
Build a development environment using Jupyter and Flask with Python in Docker (supports both VS Code / code-server)
Build a Python environment with WSL + Pyenv + Jupyter + VS Code
Create a Python environment for professionals in VS Code on Windows
Create a simple Python development environment with VS Code and Docker
How to build Python and Jupyter execution environment with VS Code
Build Python development environment (pythonz, virtualenv, direnv)
[Windows 10] [Development environment construction] How to resolve when unresolved import'django.xxx' python (unresolved-import) occurs in VS Code
VS Code + Azure Functions + Python environment construction procedure
[Python] Build a Django development environment with Docker
Build a Python development environment on your Mac
Build a Minecraft plugin development environment in Eclipse
Build a Python development environment on Raspberry Pi
Make your Python environment "easy" with VS Code
Set up TinyGo development environment for VS Code
Python local development environment construction template [Flask / Django / Jupyter with Docker + VS Code]
Framework development in Python
Slackbot development in Python
Build a GVim-based Python development environment on Windows 10 (3) GVim8.0 & Python3.6
Build an interactive environment for machine learning in Python
Revive symbol search in Python workspace with VS Code
Build a GVim-based Python development environment on Windows 10 (1) Installation
Build a Python development environment on Mac OS X
Python development environment with Windows + Anaconda3 + Visual Studio Code
Build a Python development environment using pyenv on MacOS
Python development environment with Windows + Python + PipEnv + Visual Studio Code
Expose settings.json for efficient Python coding in VS Code
I tried to build an environment with WSL + Ubuntu + VS Code in a Windows environment
[Django] Use VS Code + Remote Containers to quickly build a Django container (Docker) development environment.
Python with VS Code (Windows 10)
[No venv required] The strongest Python development environment created with Remote Containers [VS Code / Docker]
Build an Ubuntu python development environment on Google Cloud Platform
How to use VS Code in venv environment on windows
Write Spigot in VS Code
Build a Go development environment with VS Code's Remote Containers
Commands often used in the development environment during Python implementation
Create a VS Code + Docker development environment on a Linux VM
CI environment construction ~ Python edition ~
Handle environment variables in Python
Building a development environment for Android apps-creating Android apps in Python
Build python3 environment with ubuntu 16.04
Build Python environment on Windows
Debug Python with VS Code
Prepare Python development environment with Mac + Windows + VisualStudio Code (Windows version)
Build python environment with direnv
Organize your Python development environment
Prepare Python development environment with Mac + Windows + VisualStudio Code (Mac version)
Build a GVim-based Python development environment on Windows 10 (2) Basic settings
Build python environment on windows
Python development in Visual Studio 2017
[MEMO] [Development environment construction] Python
[For organizing] Python development environment
Build a Python environment offline
Specification generation and code generation in REST API development (Python edition)
I want to be able to run Python in VS Code