Build and test a CI environment for multiple versions of Python

My name is @giginet and I am in charge of the 25th day of Python Advent Calendar. It's the last day, so I managed to write it so as not to be late.

The other day, I developed a simple plugin for Django. So, when I built a CI environment to absorb the difference between Python 2 and 3 and test the implemented plug-in in multiple environments, there were many addictions, so I will summarize it at this opportunity.

This article will show you how to use Travis CI to run tests across multiple Python versions to measure coverage.

Implement the module

First, implement the simple module you want to test. This time I put calculator.py in the calculator module and implemented the Calculator class.

This time, it is assumed that all the code will be implemented in Python3.

calculator/calculator.py


class Calculator(object):
    @staticmethod
    def power(number):
        return number * number

Implement test case

Next, implement the test case. Here we are simply testing if Calculator # power works.

tests/test_calculator.py


from unittest import TestCase
from calculator.calculator import Calculator


class CalculatorTestCase(TestCase):
    def setUp(self):
        self.calculator = Calculator()

    def test_power(self):
        self.assertEqual(self.calculator.power(12), 144)

Implement Test Runner

Now it's time to explore the test cases you've written and write a script to run the tests.

If you have Python 2.7 or above, you can easily search for test cases and execute tests as follows.

$ python -m unittest discover

If you're also testing Python 2.6 and below, the standard ʻunittest module doesn't include adiscover to search for test cases, so you'll need to install a package called ʻunittest2 instead.

requirements-test.txt


unittest2
$ pip install -r requirements-test.txt

Using ʻunittest2`, I implemented a script that searches for and executes test cases as follows.

runtest.py


#!/usr/bin/env python

import unittest2 as unittest

if __name__ == "__main__":
    suite = unittest.TestLoader().discover('tests', pattern = "test_*.py")
    unittest.TextTestRunner().run(suite)

When you do this, the test will run as shown below.

$ chmod +x runtest.py
$ ./runtest.py
.
----------------------------------------------------------------------
Ran 1 tests in 0.001s

OK

Test with multiple versions of Python using tox

Next, let's run this test case in multiple environments.

** tox ** is a well-known easy way to test a source with multiple versions of Python or Django.

Welcome to the tox automation project — tox 2.3.1 documentation

tox is a tool that allows you to build a test environment by changing various conditions and execute tests in multiple environments at once.

You can build a test environment by defining a configuration file called tox.ini, so this time we will define it as follows.

tox.ini


[tox]
skipsdist = True
envlist =
    py26,
    py27,
    py33,
    py34,
    py35

[testenv]
deps=
    -rrequirements-test.txt
commands=
  ./runtest.py

Please refer to the documentation for details.

tox configuration specification — tox 2.3.1 documentation

Finally, running tox allows you to run all test environments at the same time.

This time I will run the test only with Python 3.5 to check the operation.

$ tox -r -e 'py35'

You have now built a test environment that supports multiple Python versions.

Run tox on Travis CI with tox-travis

Now let's run this on Travis CI.

This time we will use a module called tox-travis.

ryanhiebert/tox-travis

With this, you can omit the settings for running tox on Travis CI.

For example, you can run the test in multiple environments just by preparing .travis.yml as shown below.

yaml:.travis.yml


sudo: false
language: python
python:
  - 2.6
  - 2.7
  - 3.3
  - 3.4
  - 3.5

install:
  - pip install tox tox-travis
  - pip install coverage coveralls

script:
  - tox -r

When you do this on Travis CI, multiple containers will be launched for easy testing, as shown below. It's convenient.

Screen Shot 2015-12-25 at 01.30.33.png

Keep Python 2 and 3 compatible with the __future__ module

In order to run the code written for Python3 on Python2, you have to use the __future__ module etc. and write it in a backward compatible manner.

Please refer to the documentation for more details, as this article does not go into detail.

Easy, clean, reliable Python 2/3 compatibility — Python-Future documentation

Mock test with Python2

If you want to test with mock or stub, Python 3 includes a mock module in the standard library.

26.5. unittest.mock — mock object library — Python 3.5.1 documentation

On the other hand, mock cannot be used as standard in the environment before Python2.

There is a package that backports the Python3 mock module before 2, so let's introduce it.

requirements-test.txt


mock

First try to import ʻunittest.mock, and if it is not included in the standard library, import it from the mock` library.

from unittest import TestCase

try:
    from unittest.mock import MagicMock
except ImportError:
    from mock import MagicMock


class CalcTestCase(TestCase):
    def test_mock(self):
        mock_object = MagicMock()
        mock_object.__str__.return_value = 'hello'
        self.assertEqual(str(mock_object), 'hello')

This way, you can do mock tests on Python 2 as well as 3.

For details on how to use mock, refer to the following.

Python --Mock Mock ~ unittest.mock, mock.mock revisit ~ --Qiita

Measure coverage

Once the tests pass in all environments, finally measure the coverage and manage it with coverails.

First, measure the coverage by executing coverage in tox.ini as follows.

tox.ini


deps=
    -rrequirements-test.txt
    coverage
commands=
  {envbindir}/coverage run --append --source=calculator runtest.py

At this time, if you define a file called .coveragerc, you can set the rules for coverage measurement. The following defines files and lines for which coverage measurement is not performed.

See below for details.

Configuration files — Coverage.py 4.0.3 documentation

.coveragerc


[report]
include =
    calculator/*.py
omit =
    calculator/__init__.py
exclude_lines =
    pragma: no cover
    if __name__ == .__main__.:

Finally, add the following to .travis.yml so that the log will be sent to coversalls after the test is completed.

yaml:.travis.yml


install:
  - pip install tox tox-travis
  - pip install coverage coveralls

after_success:
  - coverage report
  - coveralls

You can now record coverage in Coveralls.

Screen Shot 2015-12-25 at 01.24.24.png

Summary

In this article, I've briefly summarized how to test your Python library with multiple versions.

The Test Suite introduced this time is briefly summarized in the following repository.

https://github.com/giginet/python-ci-suite

You can also use tox well to test a library with multiple versions.

The following Django plugins are testing with multiple versions of Python and a combination of Django. If you are interested, please refer to it.

giginet/django-debug-toolbar-vcs-info

Recommended Posts

Build and test a CI environment for multiple versions of Python
How to build an environment for using multiple versions of Python on Mac
Use multiple versions of python environment with pyenv
Quickly build a python environment for deep learning and data science (Windows)
Building a Docker working environment for R and Python
Build a python virtual environment with virtualenv and virtualenvwrapper
Build a python environment for each directory with pyenv-virtualenv
A quick comparison of Python and node.js test libraries
Install multiple versions of Python
Build a python environment to learn the theory and implementation of deep learning
Build a python virtual environment with virtualenv and virtualenvwrapper
Build a Python environment offline
Build a Python environment and transfer data to the server
Build a python environment on CentOS 7.7 for your home server
Environment construction of python and opencv
Build a python3 environment on CentOS7
[Building a CI environment in 2 hours] Procedure for building a Python Web server with CircleCI and passing an HTTP communication test
Building a Docker working environment for R and Python 2: Japanese support
Build a 64-bit Python 2.7 environment with TDM-GCC and MinGW-w64 on Windows 7
Build a local development environment for Lambda + Python using Serverless Framework
Build a Python environment on your Mac with Anaconda and PyCharm
Try using virtualenv, which can build a virtual environment for Python
[DynamoDB] [Docker] Build a development environment for DynamoDB and Django with docker-compose
Build an environment for Blender built-in Python
Build Python3 and OpenCV environment on Ubuntu 18.04
Build a python environment on MacOS (Catallina)
Let's create a virtual environment for Python
Connect a lot of Python or and and
I want to build a Python environment
[Mac] Building a virtual environment for Python
Build a python virtual environment with pyenv
Build a Python + OpenCV environment on Cloud9
Build a modern Python environment with Neovim
Building a Python development environment for AI development
Build a version control environment for Python, Ruby, Perl, Node.js on UNIX
Try to set up a Vim test environment quite seriously (for Python)
Build a lightweight server in Python and listen for Scratch 2 HTTP extensions
Create a Vim + Python test environment in 1 minute
Building a python environment with virtualenv and direnv
Simply build a Python 3 execution environment on Windows
Build a python environment with ansible on centos6
ffmpeg-Build a python environment and split the video
Build a Python environment on Mac (Mountain Lion)
[Python] Build a Django development environment with Docker
Create a python3 build environment with Sublime Text3
Installation of Python3 and Flask [Environment construction summary]
Developed and verified with multiple python versions with direnv
Build a Python development environment on your Mac
Build a simple Python virtual environment without pyenv
Build a virtual environment with pyenv and venv
python development environment -use of pyenv and virtualenv-
Commands for creating a python3 environment with virtualenv
Build a Kubernetes environment for development on Ubuntu
Procedure for creating a Python quarantine environment (venv environment)
Try using tensorflow ① Build python environment and introduce tensorflow
Build a Python environment with OSX El capitan
Quickly build a Python Django environment with IntelliJ
A memo for creating a python environment by a beginner
[Python] A rough understanding of iterators, iterators, and generators
A discussion of the strengths and weaknesses of Python
Build PyPy and Python execution environment with Docker