You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the python testing world the stdlib untittest testing library and runner are not the defacto standard for running tests, most of the python world uses pytest. In recent years the testing cabal stdlib unittest extension stack that we've built stestr around hasn't been super actively maintained and new features in stdlib unittest aren't supported at all. To enable broader adoption of stestr and also enable using more advanced testing features we should look into adding a mode of operation that leverages pytest as a test runner instead of our unittest based runner. We should still continue to maintain, support, and default to the current test execution model as that's what our users expect. But having a way to configure stestr to use pytest for execution instead would be a great feature. This would enable users to get all the advanced parallelism and scheduling features that stestr provides and the more sophisticated test library features that pytest offers.
The only requirement in practice to make this happen in practice is to have pytest emit a real time subunit stream as it executes tests. If we can teach pytest how to do this then in theory it's just a drop in replacement inside of stestr to call pytest instead of python -m stestr.subunit_runner.run (which is part of stestr's heritage being forked from testr that supports working like a test runner runner).
The open questions for me are on how we package this (do we make pytest a hard requirement) and include a plugin to enable subunit output natively, or do we leave the subunit plugin as an external library and make pytest an optional dependency?
Additional context
@jogo started an implementation of this last year which relied on a fork of an existing pytest-subunit plugin for adding subunit support to pytest. We can take over where he left off and figure out how to best integrate this into stestr.
This commit adds a new runner mode to stestr that enables users to run
test suites using pytest instead of unittest. This is an opt-in feature
as for compatibility reasons we'll always default to using unittest, as
all existing stestr users have only been leveraging unittest to run
tests. A pytest plugin that adds a subunit output mode is now bundled
with stestr. When the user specifies running tests with pytest it calls
out to pytest just as stestr does with the integrated unittest extension
today and sets the appropriate flags to enable subunit output.
To facilitate this new feature pytest is added to the stestr
requirements list. I debated making it an optional dependency, but to
make it easier for the significantly larger pytest user base (it's
downloaded ~900x more per month) it seemed simpler to just make it a
hard requirement. But I'm still not 100% on this decision so if there
is sufficient pushback we can start it out as an optional dependency.
Co-Authored-By: Joe Gordon <[email protected]>
Closes: #354
In the python testing world the stdlib untittest testing library and runner are not the defacto standard for running tests, most of the python world uses pytest. In recent years the testing cabal stdlib unittest extension stack that we've built stestr around hasn't been super actively maintained and new features in stdlib unittest aren't supported at all. To enable broader adoption of stestr and also enable using more advanced testing features we should look into adding a mode of operation that leverages pytest as a test runner instead of our unittest based runner. We should still continue to maintain, support, and default to the current test execution model as that's what our users expect. But having a way to configure stestr to use pytest for execution instead would be a great feature. This would enable users to get all the advanced parallelism and scheduling features that stestr provides and the more sophisticated test library features that pytest offers.
The only requirement in practice to make this happen in practice is to have pytest emit a real time subunit stream as it executes tests. If we can teach pytest how to do this then in theory it's just a drop in replacement inside of stestr to call
pytest
instead ofpython -m stestr.subunit_runner.run
(which is part of stestr's heritage being forked from testr that supports working like a test runner runner).The open questions for me are on how we package this (do we make pytest a hard requirement) and include a plugin to enable subunit output natively, or do we leave the subunit plugin as an external library and make pytest an optional dependency?
Additional context
@jogo started an implementation of this last year which relied on a fork of an existing
pytest-subunit
plugin for adding subunit support to pytest. We can take over where he left off and figure out how to best integrate this into stestr.https://github.com/jogo/pytest-subunit
https://github.com/jogo/stestr/tree/pytest
The text was updated successfully, but these errors were encountered: