Add C++ coverage to CI #265

chaeyeunpark · 2022-03-21T19:09:57Z

Add C++ code coverage to GH Action.

github-actions · 2022-03-21T19:10:16Z

Hello. You may have forgotten to update the changelog!
Please edit .github/CHANGELOG.md with:

A one-to-two sentence description of the change. You may include a small working example for new features.
A link back to this PR.
Your name (or GitHub username) in the contributors section.

github-actions · 2022-03-21T19:11:11Z

Test Report (C++) on Ubuntu

          1 files ±0           1 suites ±0 0s ⏱️ ±0s
      660 tests ±0       660 ✔️ ±0 0 💤 ±0 0 ❌ ±0
228 174 runs ±0 228 174 ✔️ ±0 0 💤 ±0 0 ❌ ±0

Results for commit 97f2cfb. ± Comparison against base commit ef2d832.

♻️ This comment has been updated with latest results.

codecov · 2022-03-21T19:12:10Z

Codecov Report

Merging #265 (2f98384) into master (ef2d832) will not change coverage.
The diff coverage is 100.00%.

❗ Current head 2f98384 differs from pull request most recent head 97f2cfb. Consider uploading reports for the commit 97f2cfb to get more accurate results

@@           Coverage Diff           @@
##           master     #265   +/-   ##
=======================================
  Coverage   99.73%   99.73%           
=======================================
  Files           4        4           
  Lines         383      383           
=======================================
  Hits          382      382           
  Misses          1        1

Impacted Files	Coverage Δ
pennylane_lightning/_version.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ef2d832...97f2cfb. Read the comment docs.

…ning into cpp_coverage

chaeyeunpark · 2022-03-22T02:41:49Z

Hi all, this PR adds C++ coverage to CodeCov. Now it only uploads results from Linux and Windows, but macOS can be also added if it is necessary. Still, it seems like the coverage results only count the lines that are visible under the given environment, i.e. #ifdef __GCC__ block is not added to the denominator of the percentage it calculates when MSVC is used.

As I also renamed a required test, GH complains a bit about checks, but we may change our GH setting when this is ready to be merged.

mlxd

Awesome work @chaeyeunpark
Left a few small comments/questions.

mlxd · 2022-03-22T13:38:51Z

.github/workflows/tests_linux.yml

@@ -98,6 +117,21 @@ jobs:
          check_name: Test Report (C++) on Ubuntu
          files: Build/tests/results/report.xml

+      - name: Build and run unit tests for code coverage
+        run: |
+            cmake pennylane_lightning/src -BBuildCov  -DCMAKE_BUILD_TYPE=Debug -DBUILD_TESTS=ON -DENABLE_COVERAGE=ON


Do we need to enable -DENABLE_BLAS=ON here also?

Oops added 👍

mlxd · 2022-03-22T13:39:50Z

.github/workflows/tests_windows.yml

+            .\Build\tests\Debug\runner.exe --order lex --reporter junit --out .\Build\tests\results\report.xml
+            OpenCppCoverage --sources pennylane_lightning\src --export_type cobertura:coverage.xml Build\tests\Debug\runner.exe


Nice work --- this must've been a fun adventure to find

Yes, but as we anyway have an AWS machine for MSVC, it wasn't that challenging 😉

mlxd · 2022-03-22T13:41:08Z

pennylane_lightning/src/util/RuntimeInfo.cpp


 namespace Pennylane::Util {
-#if defined(__x86_64__) && (defined(__GNUC__) || defined(__clang__))
+#if (defined(__GNUC__) || defined(__clang__)) && defined(__x86_64__)


I assume __clang__ is also defined on intel's latest compilers (icpx)?

Yes but as it also supports cpuid.h normally, I think it is fine (also tested locally).

mlxd · 2022-03-22T13:42:28Z

.github/workflows/tests_linux.yml

@@ -30,13 +30,15 @@ jobs:
          python-version: '3.7'

      - uses: actions/checkout@v2
+        with:
+          fetch-depth: 2 # for codecov


Nice -- wasn't aware this is an option.

Yes, Codecov raises this warning. I think this is the reason why CodeCov sometimes couldn't merge coverage data uploaded from the same commit hash.

chaeyeunpark added 5 commits March 15, 2022 15:23

Add cmake options for coverage

9b79196

Merge remote-tracking branch 'origin/master' into cpp_coverage

999ff8b

Fix tests for MSVC

75e2fe4

Add workflows

e44c9da

Rollback

245bcb3

Auto update version

3a59d91

chaeyeunpark added 3 commits March 21, 2022 15:12

Fix

506a1f5

Merge branch 'cpp_coverage' of github.com:PennyLaneAI/pennylane-light…

c34f99b

…ning into cpp_coverage

Exclude C++ tests dir itself from codecov

2a231bd

chaeyeunpark changed the title ~~Add C++ coverage~~ Add C++ coverage to CI Mar 21, 2022

chaeyeunpark and others added 14 commits March 21, 2022 15:50

Merge remote-tracking branch 'origin/master' into cpp_coverage

0e6f814

Update cmake files

b67c8c8

Update workflows

5010453

Merge remote-tracking branch 'origin/master' into cpp_coverage

5ce3de1

Auto update version

d1dd89a

Turn off openmp for windows

60cb40c

Update path format

5461255

Windows path update

a1baff4

Change to Release

1997175

Test codecov upload

e6c6fe2

Fix for opencppcoverage

1d5f492

Fix MSVC

71ebb65

Fix codecov uploader

d9a9d78

Remove unsupported option for codecov uploader v1

82594ea

chaeyeunpark added 2 commits March 21, 2022 22:43

Renamed yaml file

4dd9db4

Set fetch-depth for codecov

7a69272

chaeyeunpark requested a review from mlxd March 22, 2022 13:23

chaeyeunpark marked this pull request as ready for review March 22, 2022 13:36

mlxd reviewed Mar 22, 2022

View reviewed changes

chaeyeunpark and others added 4 commits March 22, 2022 09:50

Add ENABLE_BLAS for OpenBLAS CI

1246d7c

Merge branch 'master' into cpp_coverage

68a1ab3

Auto update version

e483032

Trigger CI

4cb6337

mlxd approved these changes Mar 23, 2022

View reviewed changes

chaeyeunpark and others added 4 commits March 23, 2022 09:11

Trigger CI

99e5b41

Auto update version

e105279

Merge remote-tracking branch 'origin/master' into cpp_coverage

6760a0a

Auto update version

97f2cfb

chaeyeunpark merged commit 0c2f64f into master Mar 23, 2022

chaeyeunpark deleted the cpp_coverage branch March 23, 2022 13:39

chaeyeunpark mentioned this pull request Mar 23, 2022

Add C++ coverage checks to Lightning #258

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add C++ coverage to CI #265

Add C++ coverage to CI #265

chaeyeunpark commented Mar 21, 2022 •

edited

Loading

github-actions bot commented Mar 21, 2022

github-actions bot commented Mar 21, 2022 •

edited

Loading

codecov bot commented Mar 21, 2022 •

edited

Loading

chaeyeunpark commented Mar 22, 2022 •

edited

Loading

mlxd left a comment

mlxd Mar 22, 2022

chaeyeunpark Mar 22, 2022

mlxd Mar 22, 2022

chaeyeunpark Mar 22, 2022

mlxd Mar 22, 2022

chaeyeunpark Mar 22, 2022 •

edited

Loading

mlxd Mar 22, 2022

chaeyeunpark Mar 22, 2022 •

edited

Loading

		.\Build\tests\Debug\runner.exe --order lex --reporter junit --out .\Build\tests\results\report.xml
		OpenCppCoverage --sources pennylane_lightning\src --export_type cobertura:coverage.xml Build\tests\Debug\runner.exe

Add C++ coverage to CI #265

Add C++ coverage to CI #265

Conversation

chaeyeunpark commented Mar 21, 2022 • edited Loading

github-actions bot commented Mar 21, 2022

github-actions bot commented Mar 21, 2022 • edited Loading

Test Report (C++) on Ubuntu

codecov bot commented Mar 21, 2022 • edited Loading

Codecov Report

chaeyeunpark commented Mar 22, 2022 • edited Loading

mlxd left a comment

Choose a reason for hiding this comment

mlxd Mar 22, 2022

Choose a reason for hiding this comment

chaeyeunpark Mar 22, 2022

Choose a reason for hiding this comment

mlxd Mar 22, 2022

Choose a reason for hiding this comment

chaeyeunpark Mar 22, 2022

Choose a reason for hiding this comment

mlxd Mar 22, 2022

Choose a reason for hiding this comment

chaeyeunpark Mar 22, 2022 • edited Loading

Choose a reason for hiding this comment

mlxd Mar 22, 2022

Choose a reason for hiding this comment

chaeyeunpark Mar 22, 2022 • edited Loading

Choose a reason for hiding this comment

chaeyeunpark commented Mar 21, 2022 •

edited

Loading

github-actions bot commented Mar 21, 2022 •

edited

Loading

codecov bot commented Mar 21, 2022 •

edited

Loading

chaeyeunpark commented Mar 22, 2022 •

edited

Loading

chaeyeunpark Mar 22, 2022 •

edited

Loading

chaeyeunpark Mar 22, 2022 •

edited

Loading