junitxml: incorrect tagging of import errors

[pytestbot]

Originally reported by: Oleg Sinyavskiy (BitBucket: otognan, GitHub: otognan)

---

Hello,

There are two kinds of failures you can get running py.test - "failure" and "error".
When you have import errors, it supposed to be an "error" instead of "failure.
However junit xml is inconsistent distinguishing those.
I ran py.test on my suite and I've got 2 import errors and 2 pep8 failures. Here is a junit xml header:

#!xml

<testsuite errors="2" failures="2" name="pytest" skips="0" tests="174" time="4.716">

However import error files are still tagged as "failures" instead of "errors":

#!xml

<testcase classname="" name="test_battery_web_plugin" time="0">
<failure message="collection failure">
btest_battery_web_plugin.py:30: in <module> from battery_web_plugin import BatteryWebService battery_web_plugin.py:24: in <module> import abap E ImportError: No module named abap
</failure>
</testcase>

This bug shows up itself during parsing with https://pypi.python.org/pypi/xunitparser.
It has an assert:

#!python

assert len(tr.errors) == int(root.attrib['errors'])

which check if header summary is consistent with parsed tests descriptions.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/575

[pytestbot]

Original comment by Oleg Sinyavskiy (BitBucket: otognan, GitHub: otognan):

---

Also, if you ignore this assert on errors, xunitparser failes in the next assert because number of tests in the header is kind of wrong: there are 174 tests, but there are 176 elements in the xml (174 successes and failures) + 2 failures that are supposed to be errors.
But its not clear whether its a pytest or xunitparser bug.

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

Merged in nicoddemus/pytest (pull request #203)

fix issue575: xunit-xml reporting collection errors as failures

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

fix issue575: xunit-xml reporting collection errors as failures

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

Merged in nicoddemus/pytest (pull request #203)

fix issue575: xunit-xml reporting collection errors as failures

[pytestbot]

Original comment by Oleg Sinyavskiy (BitBucket: otognan, GitHub: otognan):

---

Thank you guys!