Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal error "Can't combine line data with arc data" for code using subprocesses #243

Closed
skirpichev opened this issue Nov 21, 2018 · 3 comments

Comments

@skirpichev
Copy link

While updating travis-ci config of the diofant from dist:trusty to dist:xenial I noticed build failures like this for jobs with coverage profiling enabled.

Little debugging shows, that problem seems to be related with autowrap() function calls. For example, in this scenario:

$ pip install .[exports,develop]
$ cat diofant/tests/test-cov-bug.py 
from diofant.abc import x
from diofant.utilities.autowrap import autowrap
def test_1():
    assert str(x) == 'x'
def test_2():
    f = autowrap(x)
    assert f(0.1) == 0.1
$ py.test --cov diofant diofant/tests/test-cov-bug.py
========================================================= test session starts ==========================================================
platform linux -- Python 3.5.3, pytest-4.0.0, py-1.7.0, pluggy-0.8.0
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/sk/src/diofant/.hypothesis/examples')
cache: True
ground types: gmpy
rootdir: /home/sk/src/diofant, inifile: setup.cfg
plugins: xdist-1.24.1, timeout-1.3.3, forked-0.2, cov-2.6.0, hypothesis-3.82.1
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 2 items                                                                                                                      
diofant/tests/test-cov-bug.py ..                                                                                                 [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/_pytest/main.py", line 185, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/_pytest/main.py", line 225, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/pluggy/hooks.py", line 284, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/pluggy/manager.py", line 61, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/pytest_cov/engine.py", line 167, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/coverage/control.py", line 818, in combine
INTERNALERROR>     self.data, aliases=aliases, data_paths=data_paths, strict=strict,
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/coverage/data.py", line 736, in combine_parallel_data
INTERNALERROR>     data.update(new_data, aliases=aliases)
INTERNALERROR>   File "/home/sk/venv/dev/lib/python3.5/site-packages/coverage/data.py", line 488, in update
INTERNALERROR>     raise CoverageException("Can't combine line data with arc data")
INTERNALERROR> coverage.misc.CoverageException: Can't combine line data with arc data
======================================================= 2 passed in 3.66 seconds =======================================================

After such test run there are files like .coverage.hostname.1234, which left on the system:

$ cat .coverage.note.3431.337550
!coverage.py: This is a private format, don't read it directly!{"lines":{}}(dev) 

Problem related with the test_2(), if you blacklist this test by -k 'not test_2 - there will be no internal error. Also, as you can see, test_2 itself was passed, problem happens only for coverage builds.

Not sure why everything was working on Trusty's Travis builds (see e.g. this and coverage for autowrap.py seems to be enabled and around 91%), the problem happens to me for all local test runs with same CPython/packages versions as on Travis-CI.

@blueyed
Copy link
Contributor

blueyed commented Nov 22, 2018

Looks like you're mixing branch and line coverage (e.g. due to different/no config files being used in subprocesses).
Please see #237 for comments from me how to debug this.

skirpichev added a commit to skirpichev/diofant that referenced this issue Nov 22, 2018
See
pytest-dev/pytest-cov#237 (comment)

Also pytest options for Travis CI builds were consolidated
in PYTEST_ADDOPTS environment variable.
@skirpichev
Copy link
Author

@blueyed, thank you for workaround. --cov-config setup.cfg does solve problem for me.

Feel free to close this report, if it seems to be a duplicate. Anyway, I believe there is an issue.

@blueyed
Copy link
Contributor

blueyed commented Nov 22, 2018

@skirpichev
Glad it works - yes there is certainly an issue. Help with fixing this would be appreciated, of course.
Closing as duplicate.

@blueyed blueyed closed this as completed Nov 22, 2018
NickCrews added a commit to staticjinja/staticjinja that referenced this issue Dec 2, 2020
I was getting (both locally and on GitHub Actions)
Internal error "Can't combine line data with arc data"
apparently because this test code calls subprocesses and the results
are getting garbled as they are written to the single .coverage file.
pytest-dev/pytest-cov#243 (comment) has more info.
NickCrews added a commit to staticjinja/staticjinja that referenced this issue Dec 2, 2020
I was getting (both locally and on GitHub Actions)
Internal error "Can't combine line data with arc data"
apparently because this test code calls subprocesses and the results
are getting garbled as they are written to the single .coverage file.
pytest-dev/pytest-cov#243 (comment) has more info.
grantjenks pushed a commit to grantjenks/blue that referenced this issue Feb 12, 2021
Fresh setup of blue, installed requirements.txt, and ran pytest. Then got:

INTERNALERROR> coverage.misc.CoverageException: Can't combine arc data with
line data

Debugging lead to pytest-dev/pytest-cov#243 where
adding --cov-branch was identified as a workaround ¯\_(ツ)_/¯
oleksandr-pavlyk added a commit to IntelPython/dpctl that referenced this issue Sep 9, 2021
This works around coverage config discrepancy between the main
process and a subprocess. See pytest-dev/pytest-cov#243
oleksandr-pavlyk added a commit to IntelPython/dpctl that referenced this issue Sep 9, 2021
…st Cython API functions

The extension is built in temp folder.

Added cython as Cython test dependency

Pass --cov-config option to pytest with coverage.

This works around coverage config discrepancy between the main
process and a subprocess. See pytest-dev/pytest-cov#243

added _cython_api.pyx to flake exception list

Exclude dpctl/tests/* and dpctl/_version from coverage reporting

Fix loading of Cython extension _cython_api on Windows

Add test for property=int usage
oleksandr-pavlyk added a commit to IntelPython/dpctl that referenced this issue Sep 9, 2021
…st Cython API functions

The extension is built in temp folder.

Added cython as Cython test dependency

Pass --cov-config option to pytest with coverage.

This works around coverage config discrepancy between the main
process and a subprocess. See pytest-dev/pytest-cov#243

added _cython_api.pyx to flake exception list

Exclude dpctl/tests/* and dpctl/_version from coverage reporting

Fix loading of Cython extension _cython_api on Windows

Add test for property=int usage
aburrell added a commit to aburrell/apexpy that referenced this issue May 3, 2022
Added the cov-config flag at suggestion of: pytest-dev/pytest-cov#243.
EpicWink added a commit to EpicWink/proxpi that referenced this issue Dec 12, 2022
engineervix added a commit to engineervix/cookiecutter-wagtail-vix that referenced this issue Dec 27, 2023
cuinixam added a commit to cuinixam/yanga that referenced this issue Feb 23, 2024
This configuration change helps prevent coverage
job errors caused by the usage of subprocess module.
See details at pytest-dev/pytest-cov#243
cuinixam added a commit to cuinixam/yanga that referenced this issue Feb 23, 2024
This configuration change helps prevent coverage
job errors caused by the usage of subprocess module.
See details at pytest-dev/pytest-cov#243
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants