Add flag which makes pytest to return 0 exit code if no test functions were executed.

[WloHu]

Followup from #812 (comment) and I'm aware about discussion in #500 .

• a detailed description of the bug or suggestion

Title.
With this flag added calling pytest --the-flag will return 0 exit code if no test function were found.

There are 2 cases to consider:

• there is no test functions in directory,
• all tests were deselected.

I only care about the 1st scenario because my use case it to configure CI with tox where testing is one of build steps and I want to have that step in place even if there are no tests yet. Without this flag I would have to do something (e.g. add - pytest in tox.ini) and redo that after some tests were added or perform additional bash scripting which would analyze the test directory and returned code and possibly change the return code accordingly.

Also given that people from other testing tools e.g. unittest are used to scenario "if there is no tests there is no reason to fail" so this flag will give everyone an option about how pytest should behave.

The 2nd mentioned scenario can also have such option but on one side you want to treat "all tests were deselected" as a way to prevent errors. On the other side, why don't give users an option to configure pytest to their needs? Both, being arbitrary or open for options, have good reasons so it's hard for me to decide on that one.

Furthermore there may come a question "if you can configure this return code then why shouldn't it be possible to configure all of them?". I only want to mention this because it is an extention to this issue and probably requires a separate feature request.

• pytest and operating system versions

pytest=5.0.1
OS=Arch Linux (Linux-5.2.2-arch1-1-ARCH-x86_64-with-glibc2.2.5)

[RonnyPfannschmidt]

-1 on this in core, this can be in a plug-in or in a wrapper script

[nicoddemus]

Thanks @WloHu for following up.

Indeed a plugin might be the best place for this. Would that solve the problem for you @WloHu?

[nicoddemus]

Hah, in fact such a plugin already exists:

https://github.com/yashtodi94/pytest-custom_exit_code

[nicoddemus]

Mentioned the plugin in #5690 so it is more visible.

[WloHu]

@nicoddemus Yup that works. Thanks!

Honestly I didn't think about searching plugin for that because it seemed to me like close-to-the-core functionality. Anyway, I'm closing this.

[nicoddemus]

close-to-the-core functionality

pytest is suprisingly extensible. I suggest to take a look at the 31-lines plugin code if you are interested 😉

https://github.com/yashtodi94/pytest-custom_exit_code/blob/master/pytest_custom_exit_code.py

[yashtodi94]

An earlier discussion on this issue just for reference: pytest/issues#2393