Add coverage metrics from VectorCAST

[TimSVector]

Add in coverage metrics from VectorCAST.

1. Hover Over Dashboard – Added new metrics to the hover over on the main Jenkins dashboard

2. Job Main Code Coverage Trend– Added new metrics to this chart. When any new metrics are present, the chart will change from a filled graph to a line graph

3. Coverage Report Overview – Added new metrics to the coverage report. The heat map for new metrics work the same as the original metrics. Chart changes from filled to line as in Bump org.jsoup:jsoup from 1.16.1 to 1.16.2 #2 above

4. File Metrics – Added new metrics to chart. When new metrics are present, the additional metrics have new columns and column headers

5. tooltip.png – Tooltip code for hover over will display the new metrics as well.

[uhafner]

Since I don't have a project with test data (including source code) it would be helpful if you can add screenshots of the changed things in the pull request comment.

[TimSVector]

Hi Ulli, Here’s the screenshots. Please note that unless you have any of the new metrics, the views will not change and non of this will be present: 1. hoverOverDashboard.png – Added new metrics to the hover over on the main Jenkins dashboard 2. jobMainCCTrend.png – Added new metrics to this chart. When any new metrics are present, the chart will change from a filled graph to a line graph 3. covRptOverview.png – Added new metrics to the coverage report. The heat map for new metrics work the same as the original metrics. Chart changes from filled to line as in #2 4. fileMetrics.png – Added new metrics to chart. When new metrics are present, the additional metrics have new columns and column headers 5. tooltip.png – Tooltip code for hover over will display the new metrics as well. If you want to use the source code, I have it here: https://github.com/TimSVector/PointOfSales_v2.git

[TimSVector]

Not sure how to solve the Label Checker or PIT errors

[uhafner]

Hi Ulli, Here’s the screenshots.

There are no screenshots. Can you edit the description of the PR and add them there? Then everybody can see them in the ChangeLog.

[uhafner]

I will try to setup the build locally to actually see the changes...

It would be also helpful to have some tests for the source code rendering, so I will not break it in future releases.

You can also add support in the ChecksAPI (CoverageChecksPublisher) but this is optional.

[TimSVector]

1. Hover Over Dashboard – Added new metrics to the hover over on the main Jenkins dashboard

2. Job Main Code Coverage Trend– Added new metrics to this chart. When any new metrics are present, the chart will change from a filled graph to a line graph

3. Coverage Report Overview – Added new metrics to the coverage report. The heat map for new metrics work the same as the original metrics. Chart changes from filled to line as in Bump org.jsoup:jsoup from 1.16.1 to 1.16.2 #2 above

4. File Metrics – Added new metrics to chart. When new metrics are present, the additional metrics have new columns and column headers

5. tooltip.png – Tooltip code for hover over will display the new metrics as well.

[uhafner]

I think we can ignore the PIT error, it is caused by a PIT upgrade in main.

[TimSVector]

I added a new test file for statement+branch, statement+MCDC, and statement+branch+function call (includes function coverage) into VectorCastCoverageChecksPublisherTest.java which is derived from CoverageChecksPublisherTest.java.

Still getting no Method Coverage

[uhafner]

I added a new test file for statement+branch, statement+MCDC, and statement+branch+function call (includes function coverage) into VectorCastCoverageChecksPublisherTest.java which is derived from CoverageChecksPublisherTest.java.

Still getting no Method Coverage

You mean in the overview? The overview shows only value metrics. Details are available as tooltip only. The checks publisher does not show them at all. I can make that overridable if required?

[TimSVector]

Method coverage was displaying before we removed it for coverages that didn't contain MCDC/FCC coverage

3 points there the method coverage is missing or incorrect, as MC/DC Pair and/or Function Call coverage should trigger Method Coverage to be displayed

1. No method coverage showing in the Total coverage overview
2. No method coverage tab in the top bar
3. Method coverage showing as 0% in the graph
4. *not shown, but no Method Coverage showing in the Files tab.

---

Also, the console log, you can see Function Coverage in the input to the plugin, but no Method coverage in plugin processing:

Output from VectorCAST script:

lines: 79.93% (235 out of 294)
statements: 79.36% (223 out of 281)
branches: 66.18% (180 out of 272)
functions: 70.00% (21 out of 30)
function calls: 78.48% (62 out of 79)
mcdc pairs: 40.68% (24 out of 59)
coverage: 79.36% of statements
complexity: 100

Output from Plugin:

[Coverage] MODULE: 100.00% (1/1)
[Coverage] PACKAGE: 100.00% (5/5)
[Coverage] FILE: 75.00% (6/8)
[Coverage] CLASS: 75.00% (6/8)
[Coverage] LINE: 79.93% (235/294)
[Coverage] BRANCH: 66.18% (180/272)
[Coverage] MCDC_PAIR: 40.68% (24/59)
[Coverage] FUNCTION_CALL: 78.48% (62/79)
[Coverage] COMPLEXITY: 100
[Coverage] COMPLEXITY_DENSITY: 100/294
[Coverage] LOC: 294

[uhafner]

I fixed the display of the method coverage in the checks result.

On the tabs this metric is not shown intentionally as it is an aggregated metric and not a value metric. It should be visible in the bar chart though, I need to create a build with this chart...

[uhafner]

Your test case shouldParseFileWithVectorCast produces this chart:

It seems that the method coverage is shown.

[TimSVector]

It seems that the method coverage is shown.

Where would that image be when I run locally?

[uhafner]

It seems that the method coverage is shown.

Where would that image be when I run locally?

Add a breakpoint after buildWithResult. Then in the console you will find something like:

Connected to the target VM, address: '127.0.0.1:50392', transport: 'socket'
   0.058 [id=1]	INFO	o.jvnet.hudson.test.WarExploder#explode: Picking up existing exploded jenkins.war at /Users/hafner/git/warnings-ng-plugin-devenv/coverage-plugin/plugin/target/jenkins-for-test
   0.419 [id=1]	INFO	o.jvnet.hudson.test.JenkinsRule#createWebServer: Running on http://localhost:50398/jenkins/
   0.829 [id=48]	INFO	jenkins.InitReactorRunner$1#onAttained: Started initialization
   4.774 [id=51]	INFO	jenkins.InitReactorRunner$1#onAttained: Listed all plugins
   4.815 [id=54]	INFO	j.b.api.BouncyCastlePlugin#start: 

Click on the link and you will see the Jenkins under test. (Make sure that you enable the breakpoint option Suspend Thread, otherwise the UI is halted as well).

[TimSVector]

I see it in debug. I'm going to create a new project in Jenkins and see if that helps.

I would like to see Method coverage when there is Function Call and/or MCDC Pairs.

[uhafner]

I created a release of the coverage model now. There is also a bugfix in this release for the merge operations (https://issues.jenkins.io/browse/JENKINS-73352).

Would it be ok, if I release this PR right now and we can look afterwards in followup PRs on how to configure other UI elements differently?

[TimSVector]

Yes – that would be fine From: Ullrich Hafner ***@***.***> Sent: Wednesday, June 26, 2024 5:06 AM To: jenkinsci/coverage-plugin ***@***.***> Cc: Schneider, Tim ***@***.***>; Author ***@***.***> Subject: Re: [jenkinsci/coverage-plugin] Add coverage metrics from VectorCAST (PR #153) I created a release of the coverage model now. There is also a bugfix in this release for the merge operations (https://issues.jenkins.io/browse/JENKINS-73352). Would it be ok, if I release this PR right now and we can look afterwards in followup PRs on how to configure other UI elements differently? — Reply to this email directly, view it on GitHub<#153 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AENYLQ6GQ2IQHZGDU2RNE33ZJJ77JAVCNFSM6AAAAABH2ZMPFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJRGE4TEMZZHA>. You are receiving this because you authored the thread.Message ID: ***@***.***>