Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore block element for BullsEye coverage reports (since BullsEye covxml 8.20.0 version) #1977

Closed
slebedev opened this issue Nov 16, 2020 · 4 comments · Fixed by #1978
Closed
Assignees
Milestone

Comments

@slebedev
Copy link

slebedev commented Nov 16, 2020

Description

Coverage sensor stops working with latest BullsEye reports. After quick investigation I've found the issue in new "block" element BullsEye generates in xml report (see current covxml doc):

Block Element BullsEye CovXML

Steps to reproduce the problem

Please provide the steps required to reproduce the problem

  1. Collect BullsEye xml coverage (8.20.2)

  2. Start scanner for code with provided coverage report

P.S. Unfortunately I can't provide an example right now.

Expected behavior

Coverage Sensor works fine and parses coverage report

Actual behavior

Scanner can't parse coverage report with block elements

Known workarounds

Removing all lines with new block elements before scanner run, i.e. like this:

sed -i '/<block line=/d' .coverage/coverage.xml

LOG file

09:32:39.790 INFO: Sensor C++ (Community) CoverageSensor [cxx]
09:32:39.791 INFO: Searching coverage reports by path with basedir '/usr/src' and search prop 'sonar.cxx.coverage.reportPath'
09:32:39.792 INFO: Searching for coverage reports '[.coverage/coverage.xml]'
09:32:39.792 INFO: Coverage BaseDir '/usr/src'
09:32:39.792 DEBUG: Parsing unit test coverage reports
09:32:39.796 DEBUG: Normalized report includes to '[/usr/src/.coverage/coverage.xml]'
09:32:39.796 DEBUG: Scanner uses normalized report path(s): '/usr/src/.coverage/coverage.xml'
09:32:39.822 INFO: Parser will parse '1' report file(s)
09:32:39.823 DEBUG: Parsing 'Cobertura' format
09:32:40.879 DEBUG: Report is empty {}
org.sonar.cxx.sensors.utils.EmptyReportException: Coverage report /usr/src/.coverage/coverage.xml result is empty (parsed by CoberturaParser)
	at org.sonar.cxx.sensors.coverage.CxxCoverageSensor.parseCoverageReport(CxxCoverageSensor.java:89)
	at org.sonar.cxx.sensors.coverage.CxxCoverageSensor.processReports(CxxCoverageSensor.java:135)
	at org.sonar.cxx.sensors.coverage.CxxCoverageSensor.executeImpl(CxxCoverageSensor.java:122)
	at org.sonar.cxx.sensors.utils.CxxReportSensor.execute(CxxReportSensor.java:253)
	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:59)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:77)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:59)
	at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
	at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:400)
	at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:395)
	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:358)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
	at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:141)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:73)
	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:67)
	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
	at com.sun.proxy.$Proxy0.execute(Unknown Source)
	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
	at org.sonarsource.scanner.cli.Main.main(Main.java:61)

09:32:40.879 DEBUG: Parsing 'Bullseye' format
09:32:41.356 INFO: ------------------------------------------------------------------------
09:32:41.356 INFO: EXECUTION FAILURE
09:32:41.356 INFO: ------------------------------------------------------------------------
09:32:41.357 INFO: Total time: 7:00.405s
09:32:41.412 INFO: Final Memory: 13M/57M
09:32:41.412 INFO: ------------------------------------------------------------------------
09:32:41.412 ERROR: Error during SonarScanner execution
java.lang.NullPointerException
	at org.sonar.cxx.sensors.coverage.BullseyeParser.updateMeasures(BullseyeParser.java:207)
	at org.sonar.cxx.sensors.coverage.BullseyeParser.probWalk(BullseyeParser.java:146)
	at org.sonar.cxx.sensors.coverage.BullseyeParser.funcWalk(BullseyeParser.java:153)
	at org.sonar.cxx.sensors.coverage.BullseyeParser.fileWalk(BullseyeParser.java:161)
	at org.sonar.cxx.sensors.coverage.BullseyeParser.recTreeWalk(BullseyeParser.java:179)
	at org.sonar.cxx.sensors.coverage.BullseyeParser.recTreeWalk(BullseyeParser.java:186)
	at org.sonar.cxx.sensors.coverage.BullseyeParser.collectCoverage2(BullseyeParser.java:134)
	at org.sonar.cxx.sensors.coverage.BullseyeParser.lambda$processReport$1(BullseyeParser.java:88)
	at org.sonar.cxx.sensors.utils.StaxParser.parse(StaxParser.java:133)
	at org.sonar.cxx.sensors.utils.StaxParser.parse(StaxParser.java:101)
	at org.sonar.cxx.sensors.utils.StaxParser.parse(StaxParser.java:87)
	at org.sonar.cxx.sensors.coverage.BullseyeParser.processReport(BullseyeParser.java:92)
	at org.sonar.cxx.sensors.coverage.CxxCoverageSensor.parseCoverageReport(CxxCoverageSensor.java:83)
	at org.sonar.cxx.sensors.coverage.CxxCoverageSensor.processReports(CxxCoverageSensor.java:135)
	at org.sonar.cxx.sensors.coverage.CxxCoverageSensor.executeImpl(CxxCoverageSensor.java:122)
	at org.sonar.cxx.sensors.utils.CxxReportSensor.execute(CxxReportSensor.java:253)
	at org.sonar.scanner.sensor.AbstractSensorWrapper.analyse(AbstractSensorWrapper.java:48)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:85)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.lambda$execute$1(ModuleSensorsExecutor.java:59)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.withModuleStrategy(ModuleSensorsExecutor.java:77)
	at org.sonar.scanner.sensor.ModuleSensorsExecutor.execute(ModuleSensorsExecutor.java:59)
	at org.sonar.scanner.scan.ModuleScanContainer.doAfterStart(ModuleScanContainer.java:82)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
	at org.sonar.scanner.scan.ProjectScanContainer.scan(ProjectScanContainer.java:400)
	at org.sonar.scanner.scan.ProjectScanContainer.scanRecursively(ProjectScanContainer.java:395)
	at org.sonar.scanner.scan.ProjectScanContainer.doAfterStart(ProjectScanContainer.java:358)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
	at org.sonar.scanner.bootstrap.GlobalContainer.doAfterStart(GlobalContainer.java:141)
	at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:136)
	at org.sonar.core.platform.ComponentContainer.execute(ComponentContainer.java:122)
	at org.sonar.batch.bootstrapper.Batch.doExecute(Batch.java:73)
	at org.sonar.batch.bootstrapper.Batch.execute(Batch.java:67)
	at org.sonarsource.scanner.api.internal.batch.BatchIsolatedLauncher.execute(BatchIsolatedLauncher.java:46)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherProxy.invoke(IsolatedLauncherProxy.java:60)
	at com.sun.proxy.$Proxy0.execute(Unknown Source)
	at org.sonarsource.scanner.api.EmbeddedScanner.doExecute(EmbeddedScanner.java:189)
	at org.sonarsource.scanner.api.EmbeddedScanner.execute(EmbeddedScanner.java:138)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:112)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:75)
	at org.sonarsource.scanner.cli.Main.main(Main.java:61)

Related information

  • cxx plugin 1.3.2
  • SonarQube 7.9LTS
@guwirth guwirth added this to the 2.0.0 milestone Nov 16, 2020
@guwirth
Copy link
Collaborator

guwirth commented Nov 16, 2020

Hi @slebedev,

Thanks a lot for your feedback.

Two questions:

  • Could you provide a small XML sample please?
  • Do you think we should just ignore the block like you do or do we have to handle it? What should we do if block and probe element are both available?

Regards,

@slebedev
Copy link
Author

slebedev commented Nov 17, 2020

Hi @guwirth,

Thanks for the fast feedback!

  • Could you provide a small XML sample please?
    Sure, please, review my sample at the end of this message

  • Do you think we should just ignore the block like you do or do we have to handle it? What should we do if block and probe element are both available?
    As I understand, block element provides additional coverage information that could be ignored unless we have a restriction to handle it somehow. I don't have such restriction and propose to ignore these elements during scan.

Let's imagine that we have the following simple code:

#include <common/token.h>

namespace Core
{
  /// @brief interface for handling tokens
  class TokenHandler
  {
  public:
    /// @brief virtual destructor
    virtual ~TokenHandler() {}

    /// @brief called when handler is about to handle token
    virtual void OnHandle() = 0;

    /**
      @brief called when a token is handled
      @param token token handled
    */
    virtual void OnTokenHandled(const Common::Token& token) = 0;

    /// @brief called when parsing finished
    virtual void OnFinish() = 0;
  };
}

that produces the following coverage.xml after testing:

<?xml version="1.0" encoding="UTF-8"?>
<!-- BullseyeCoverage XML 8.20.2 Windows x64 -->
<BullseyeCoverage name="coverage.cov" dir="g:/results" buildId="745ae38_2020-11-09_19:52:34" version="6" xmlns="https://www.bullseye.com/covxml"
 fn_cov="1" fn_total="1" cd_cov="0" cd_total="0" d_cov="0" d_total="0">
    <folder name="g:" fn_cov="1" fn_total="1" cd_cov="0" cd_total="0" d_cov="0" d_total="0">
        <folder name="src" fn_cov="1" fn_total="1" cd_cov="0" cd_total="0" d_cov="0" d_total="0">
            <src name="token_handler.h" mtime="1604936026" fn_cov="1" fn_total="1" cd_cov="0" cd_total="0" d_cov="0" d_total="0">
                <fn name="Core::TokenHandler::~TokenHandler()" fn_cov="1" fn_total="1" cd_cov="0" cd_total="0" d_cov="0" d_total="0">
                    <probe line="11" column="7" kind="function" event="full"/>
                    <block line="11" entered="1"/>
                </fn>
            </src>
        </folder>
    </folder>
</BullseyeCoverage>

Coverage Sensor fails the scan for now, so my point is to not fail the whole scan if block element appears - let's just ignore it.

@slebedev slebedev changed the title Support block element for BullsEye coverage reports (since BullsEye covxml 8.20.0 version) Ignore block element for BullsEye coverage reports (since BullsEye covxml 8.20.0 version) Nov 17, 2020
@guwirth
Copy link
Collaborator

guwirth commented Nov 17, 2020

Hi @slebedev,

thanks for sharing this. Looking to our code

You see that the line below will mark line 11 as covered.

<probe line="11" column="7" kind="function" event="full"/>

Think the line below is doing the same

<block line="11" entered="1"/>

What I still don't understand: What is the advantage of this additional block tag?

Docu is saying: The block element is provided to assist with importation to a product that accepts statement coverage or basic block coverage, but not the condition/decision coverage provided by the probe element. You can ignore this element unless you have such a restriction. This coverage information is not statement coverage and is not appropriate for projects with an official mandate for statement coverage.

So I would agree to add support to the XML parser and ignore the content.

Regards,

guwirth added a commit to guwirth/sonar-cxx that referenced this issue Nov 17, 2020
- https://www.bullseye.com/help/ref-covxml.html
  - File Version: 6, Software Version: 8.20.0, Date: Oct 2020, Change: Add block element
- XML parser supports block tag
- ignore data of block tag
- close SonarOpenCommunity#1977
@guwirth guwirth self-assigned this Nov 17, 2020
@guwirth
Copy link
Collaborator

guwirth commented Nov 17, 2020

Hi @slebedev, issue is closed with #1978 for 2.0 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

2 participants