Noisy CoverageWarnings No source for code in pep517-build-env

[jaraco]

Summary

Sometime in the past year, pytest-cov started emitting lots of CoverageWarnings for files unrelated to the project under test, something like:

/Users/jaraco/code/main/tempora/.tox/python/lib/python3.10/site-packages/coverage/report.py:87: CoverageWarning: Couldn't parse '/private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pep517-build-env-c14y8sp8/lib/python3.10/site-packages/wheel/wheelfile.py': No source for code: '/private/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/pep517-build-env-c14y8sp8/lib/python3.10/site-packages/wheel/wheelfile.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")

I work with hundreds of projects that derive from jaraco/skeleton, where I reported the issue in jaraco/skeleton#56. An example project affected by this noise is jaraco/tempora.

Expected vs actual result

Simply run tox on that small project, and there's a 166 of those warnings. Since these files are unrelated to the project and since these warnings aren't emitted when running coverage natively, I'd expect these warnings not to emit when running using pytest-cov.

In addition to the warnings about pep517-build-env, there's one additional about a file called setup.py in the root of the project, which also doesn't exist. It would be nice to fix both, but this report focuses mainly on those created by files in the pep517-build-env.

Reproducer

$ git clone https://github.com/jaraco/tempora
$ cd tempora
$ git checkout 06ca5a9
$ tox

Note that the errors happen whether tox is invoked or not. Just using tox to set up the environment and then running pytest produces the same warnings. See jaraco/skeleton#56 for more details on the output in both cases.

Versions

Here's the current output of tox and pytest:

tempora main $ tox --notest
python create: /Users/jaraco/code/main/tempora/.tox/python
python develop-inst: /Users/jaraco/code/main/tempora
python installed: attrs==21.4.0,backports.unittest-mock==1.5,black==22.3.0,click==8.1.3,coverage==6.4.1,docutils==0.18.1,filelock==3.7.1,flake8==4.0.1,freezegun==1.2.1,importlib-metadata==4.11.4,iniconfig==1.1.1,jaraco.context==4.1.1,jaraco.functools==3.5.0,mccabe==0.6.1,more-itertools==8.13.0,mypy==0.960,mypy-extensions==0.4.3,packaging==21.3,pathspec==0.9.0,pep517==0.12.0,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,pycodestyle==2.8.0,pyflakes==2.4.0,pyparsing==3.0.9,pytest==7.1.2,pytest-black==0.3.12,pytest-checkdocs==2.7.1,pytest-cov==3.0.0,pytest-enabler==1.2.1,pytest-flake8==1.1.1,pytest-freezegun==0.4.2,pytest-mypy==0.9.1,python-dateutil==2.8.2,pytz==2022.1,six==1.16.0,-e git+gh://jaraco/tempora@06ca5a9e53a884856dcec160c71bfa9403f2d2d6#egg=tempora,toml==0.10.2,tomli==2.0.1,types-freezegun==1.1.9,types-pytz==2021.3.8,typing_extensions==4.2.0,zipp==3.8.0
___________________________________________________________________ summary ____________________________________________________________________
  python: skipped tests
  congratulations :)
tempora main $ .tox/python/bin/pytest
============================================================= test session starts ==============================================================
platform darwin -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
rootdir: /Users/jaraco/code/main/tempora, configfile: pytest.ini
plugins: enabler-1.2.1, backports.unittest-mock-1.5, freezegun-0.4.2, checkdocs-2.7.1, black-0.3.12, flake8-1.1.1, mypy-0.9.1, cov-3.0.0
collected 69 items                                                                                                                             

conftest.py ....                                                                                                                         [  5%]
. .                                                                                                                                      [  7%]
docs/conf.py ...                                                                                                                         [ 11%]
tempora/__init__.py ...............                                                                                                      [ 33%]
tempora/schedule.py ......                                                                                                               [ 42%]
tempora/timing.py ........                                                                                                               [ 54%]
tempora/utc.py ....                                                                                                                      [ 60%]
tempora/tests/test_schedule.py ......................                                                                                    [ 92%]
tempora/tests/test_timing.py .....                                                                                                       [100%]/Users/jaraco/code/main/tempora/.tox/python/lib/python3.10/site-packages/coverage/report.py:87: CoverageWarning: Couldn't parse '/Users/jaraco/code/main/tempora/setup.py': No source for code: '/Users/jaraco/code/main/tempora/setup.py'. (couldnt-parse)
  coverage._warn(msg, slug="couldnt-parse")
...

Config

See the jaraco/skeleton repo for the generic config used across projects or look at jaraco/tempora for a materialized project.

I plan to merge jaraco/skeleton#60 to work around the issue, but it would be nice if the issue could be addressed at the cause and not at the symptom.

[jaraco]

I notice that the warnings go away if I run with -p no:checkdocs. It seems the act of loading the project metadata in a pytest plugin is tripping up file discovery for pytest-cov. That does explain why the issue emerged last year, when that project updated its logic for loading metadata. But that doesn't explain why the warnings don't appear when running with native coverage.

[jaraco]

This issue no longer seems to be a problem. The repro no longer reproduces the issue, either because the pytest-checkdocs has evolved further or something has been fixed. Regardless, I'm going to go ahead and close this issue and back out the workaround.