Skip to content

Commit

Permalink
Extend the incremental marker for parametrize
Browse files Browse the repository at this point in the history
The incremental marker is adapted to handle properly test classes with parametrize defined at class level.

Fix #3125
  • Loading branch information
sdementen authored and nicoddemus committed Jan 29, 2020
1 parent a9eab07 commit d848a20
Showing 1 changed file with 34 additions and 6 deletions.
40 changes: 34 additions & 6 deletions doc/en/example/simple.rst
Original file line number Diff line number Diff line change
Expand Up @@ -461,21 +461,49 @@ an ``incremental`` marker which is to be used on classes:
# content of conftest.py
import pytest
# store history of failures per test class name and per index in parametrize (if parametrize used)
_test_failed_incremental: Dict[str, Dict[Tuple[int, ...], str]] = {}
def pytest_runtest_makereport(item, call):
if "incremental" in item.keywords:
# incremental marker is used
if call.excinfo is not None:
parent = item.parent
parent._previousfailed = item
# the test has failed
# retrieve the class name of the test
cls_name = str(item.cls)
# retrieve the index of the test (if parametrize is used in combination with incremental)
parametrize_index = (
tuple(item.callspec.indices.values())
if hasattr(item, "callspec")
else ()
)
# retrieve the name of the test function
test_name = item.originalname or item.name
# store in _test_failed_incremental the original name of the failed test
_test_failed_incremental.setdefault(cls_name, {}).setdefault(
parametrize_index, test_name
)
def pytest_runtest_setup(item):
if "incremental" in item.keywords:
previousfailed = getattr(item.parent, "_previousfailed", None)
if previousfailed is not None:
pytest.xfail("previous test failed ({})".format(previousfailed.name))
# retrieve the class name of the test
cls_name = str(item.cls)
# check if a previous test has failed for this class
if cls_name in _test_failed_incremental:
# retrieve the index of the test (if parametrize is used in combination with incremental)
parametrize_index = (
tuple(item.callspec.indices.values())
if hasattr(item, "callspec")
else ()
)
# retrieve the name of the first test function to fail for this class name and index
test_name = _test_failed_incremental[cls_name].get(parametrize_index, None)
# if name found, test has failed for the combination of class name & test name
if test_name is not None:
pytest.xfail("previous test failed ({})".format(test_name))
These two hook implementations work together to abort incremental-marked
tests in a class. Here is a test module example:
Expand Down

0 comments on commit d848a20

Please sign in to comment.