How to not run some tests by default using pytest's mark

Status

--I write tests in pytest that I don't want to run every time (end-to-end tests, load tests, etc.). --It is convenient if you can execute the required test with just the pytest command with no arguments.

problem

--If you don't add an argument to pytest, all the tests will be executed. ――When you select and execute a load test etc., I'm happy if it is a simple argument.

Solution

Use pytest.mark to mark (mark) tests that you do not normally run. In pytest.ini, set not to run the marked test.

some_test.py:

import pytest

def test_something():
    ...

def test_another():
    ...

@pytest.mark.performance
def test_performance():
    ...

pytest.ini:

[pytest]
...
addopts = -m "not performance"

When run with this, by default it does not run tests marked performance.

> pipenv run pytest -v
========================================= test session starts =========================================
(Omission)
collected 3 items / 1 deselected / 2 selected

test/some_test.py::test_something PASSED 
test/some_test.py::test_another PASSED

To run the marked test (only), use the -m argument on the command line.

> pipenv run pytest -v -m performance
========================================= test session starts =========================================
(Omission)
collected 3 items / 2 deselected / 1 selected

test/some_test.py::test_perfomance PASSED

Supplement

Note that even when specifying individual test files or methods directly from the command line (node ID specification), they will be ignored unless the -m option is added.

You can specify any name in @ pytest.mark, so if you typo here, it will be a test that will never be executed. As a preventive measure, write markers in pytest.ini and specify --strict in addopts.

pytest.ini:

[pytest]
...
addopts = -m "not performance" --strict
markers =
    performance: performance tests

If you mark it incorrectly, you will get an error at run time.

> pipenv run pytest -v
========================================= test session starts =========================================
(Omission)
collected 0 items / 1 error

=============================================== ERRORS ================================================
_________________________________ ERROR collecting test/some_test.py __________________________________
'peformance' not found in `markers` configuration option
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================== 1 error in 0.12s ===========================================

If you want to use different marks, it may be convenient to set an alias on the command line (shell). Complementation also works.

% alias
ppytest='pipenv run pytest'
ppytestperf='pipenv run pytest -m performance'
ppyteste2e='pipenv run pytest -m e2e'

reference

Marking test functions with attributes - pytest documentation

Configuration - pytest documentation

Test Driven Python (Amazon)

Recommended Posts

How to not run some tests by default using pytest's mark
How to run some script regularly in Django
How to run tests in bulk with Python unittest
How to not load images when using PhantomJS with Selenium
How to run Notepad ++ Python
How to install python using anaconda
[EC2] How to run selenium webdriver
How to run matplotlib on heroku
How to run TensorFlow 1.0 code in 2.0
[Rails] How to detect radical images by analyzing images using Cloud Vision API