Empty results generating report for Octokit.Net #188

StanleyGoldman · 2018-09-07T12:43:30Z

👋 Greetings.

I've been playing with coverlet and I am attempting to add coverage for GitHub Octokit.NET. Currently my attempts to generate coverage result in an empty report and no error. I'm using coverlet in another project and haven't had this problem...

$ coverlet.exe ./Octokit.Tests/bin/Debug/netcoreapp2.0/Octokit.Tests.dll --target "dotnet" --targetargs "test -c Release ./Octokit.Tests/Octokit.Tests.csproj --no-build" --format opencover --output ./coverage.xml --include [*]*
Test run for C:\Users\Spade\Projects\GitHub\octokit.net\Octokit.Tests\bin\Release\netcoreapp2.0\Octokit.Tests.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.8.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
[xUnit.net 00:00:05.1505440]     Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled [SKIP]
Skipped  Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled

Total tests: 3185. Passed: 3184. Failed: 0. Skipped: 1.
Test Run Successful.
Test execution time: 6.9781 Seconds
Test run for C:\Users\Spade\Projects\GitHub\octokit.net\Octokit.Tests\bin\Release\net452\Octokit.Tests.dll(.NETFramework,Version=v4.5.2)
Microsoft (R) Test Execution Command Line Tool Version 15.8.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
[xUnit.net 00:00:03.7761238]     Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled [SKIP]
Skipped  Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled

Total tests: 3189. Passed: 3188. Failed: 0. Skipped: 1.
Test Run Successful.
Test execution time: 6.1141 Seconds


Calculating coverage result...
  Generating report '.\coverage.xml'

+--------+--------+--------+--------+
| Module | Line   | Branch | Method |
+--------+--------+--------+--------+

What steps should I take to start debugging this issue?

The text was updated successfully, but these errors were encountered:

tonerdo · 2018-09-07T16:25:29Z

Hey @StanleyGoldman, I'll be making a new release of Coverlet in a few of hours which should massively improve performance and could potentially fix this issue. If the new release doesn't fix it then I'll take a much deeper look into this

tonerdo · 2018-09-07T18:04:49Z

@StanleyGoldman here's the latest version of the console tool: https://www.nuget.org/packages/coverlet.console/1.2.0

Also I noticed you're specifying --include [*]* to include everything which is the default case. Try removing that and seeing if it works

StanleyGoldman · 2018-09-07T19:23:59Z

Hey @tonerdo. Thanks for the quick reply.

I updated and tested, unfortunately, I am getting the same result.

15:21 $ coverlet.exe -v
Cross platform .NET Core code coverage tool
1.2.0.0

15:21 $ coverlet.exe ./Octokit.Tests/bin/Debug/netcoreapp2.0/Octokit.Tests.dll --target "dotnet" --targetargs "test -c Release ./Octokit.Tests/Octokit.Tests.csproj --no-build" --format opencover --output ./coverage.xml
Test run for C:\Users\Spade\Projects\GitHub\octokit.net\Octokit.Tests\bin\Release\netcoreapp2.0\Octokit.Tests.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.8.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
[xUnit.net 00:00:02.7443289]     Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled [SKIP]
Skipped  Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled

Total tests: 3185. Passed: 3184. Failed: 0. Skipped: 1.
Test Run Successful.
Test execution time: 6.2994 Seconds
Test run for C:\Users\Spade\Projects\GitHub\octokit.net\Octokit.Tests\bin\Release\net452\Octokit.Tests.dll(.NETFramework,Version=v4.5.2)
Microsoft (R) Test Execution Command Line Tool Version 15.8.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
[xUnit.net 00:00:04.1576116]     Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled [SKIP]
Skipped  Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled

Total tests: 3189. Passed: 3188. Failed: 0. Skipped: 1.
Test Run Successful.
Test execution time: 5.2192 Seconds


Calculating coverage result...
  Generating report '.\coverage.xml'

+--------+--------+--------+--------+
| Module | Line   | Branch | Method |
+--------+--------+--------+--------+

StanleyGoldman · 2018-09-07T21:44:03Z

I got it integrated in our build process using the msbuild route and it's producing the same output of..

+--------+--------+--------+--------+
| Module | Line   | Branch | Method |
+--------+--------+--------+--------+

https://ci.appveyor.com/project/github-windows/octokit-net/build/0.32.0-PullRequest.1866+8.build.2855

tonerdo · 2018-09-07T21:53:24Z

Do you have a branch that I can pull and test locally?

StanleyGoldman · 2018-09-08T00:46:24Z

This branch should do it for you...

https://github.com/octokit/octokit.net/tree/code-coverage

tonerdo · 2018-09-09T23:18:35Z

@StanleyGoldman, I cloned this locally and tested it out. It turns out that none of the assemblies were instrumented even though it did find instrumentable assemblies. Will take a deeper look at it

@pjanotti looping you in on this

pjanotti · 2018-09-10T01:58:16Z

Hi @StanleyGoldman - I took a quick look and the issue is that the projects use embedded pdbs and the CanInstrument() test in coverlet fails in such case. I was able to get coverage by changing the DebugType to Portable instead of embedded. A small thing to notice is that in the case of the command running against two different TFMs (netcoreapp2.0 and net452) require two distinct runs of coverlet since each one will instrument the dlls on the proper folder, e.g.:

> ..\coverlet.console\coverlet.exe ./Octokit.Tests/bin/Debug/net452/Octokit.Tests.dll --target "dotnet" --targetargs "test -c Debug -f net452 ./Octokit.Tests/Octokit.Tests.csproj --no-build" --format opencover --output ./coverage.xml
Test run for C:\s\tmp\octokit.net\Octokit.Tests\bin\Debug\net452\Octokit.Tests.dll(.NETFramework,Version=v4.5.2)
Microsoft (R) Test Execution Command Line Tool Version 15.9.0-preview-20180807-05
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
[xUnit.net 00:00:04.9773930]     Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled [SKIP]
Skipped  Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled

Total tests: 3189. Passed: 3188. Failed: 0. Skipped: 1.
Test Run Successful.
Test execution time: 5.8890 Seconds


Calculating coverage result...
  Generating report '.\coverage.xml'

+------------------+--------+--------+--------+
| Module           | Line   | Branch | Method |
+------------------+--------+--------+--------+
| Octokit          | 70.2%  | 62.8%  | 64.7%  |
+------------------+--------+--------+--------+
| Octokit.Reactive | 88.2%  | 78.3%  | 87.3%  |
+------------------+--------+--------+--------+


[Sun 09/09/2018 18:53:17.54] [C:\s\tmp\octokit.net]
> ..\coverlet.console\coverlet.exe ./Octokit.Tests/bin/Debug/netcoreapp2.0/Octokit.Tests.dll --target "dotnet" --targetargs "test -c Debug -f netcoreapp2.0 ./Octokit.Tests/Octokit.Tests.csproj --no-build" --format opencover --output ./coverage.xml
Test run for C:\s\tmp\octokit.net\Octokit.Tests\bin\Debug\netcoreapp2.0\Octokit.Tests.dll(.NETCoreApp,Version=v2.0)
Microsoft (R) Test Execution Command Line Tool Version 15.9.0-preview-20180807-05
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
[xUnit.net 00:00:05.1631499]     Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled [SKIP]
Skipped  Octokit.Tests.Reactive.ObservableRepositoriesClientTests+TheGetAllForCurrentMethod.StopsMakingNewRequestsWhenTakeIsFulfilled

Total tests: 3185. Passed: 3184. Failed: 0. Skipped: 1.
Test Run Successful.
Test execution time: 6.6424 Seconds


Calculating coverage result...
  Generating report '.\coverage.xml'

+------------------+--------+--------+--------+
| Module           | Line   | Branch | Method |
+------------------+--------+--------+--------+
| Octokit          | 70.5%  | 63.6%  | 65%    |
+------------------+--------+--------+--------+
| Octokit.Reactive | 88.2%  | 78.3%  | 87.3%  |
+------------------+--------+--------+--------+

@tonerdo I'm new to Mono.Cecil but I suspect that somehow we can defer the pdb availability test to Mono.Cecil itself and perhaps also accept something like _NT_SYMBOL_PATH to not require the pdb to be on the same folder as the dll

StanleyGoldman · 2018-09-10T02:28:24Z

@pjanotti Thanks. I thought I tried that but I apparently got turned around.

@tonerdo @pjanotti thanks for your help.

tonerdo · 2018-09-10T08:53:08Z

Thanks for your help @pjanotti, I should probably add a note about portable PDBs in the README until we can get Coverlet to work with any assembly that has debug information regardless of where it is

StanleyGoldman mentioned this issue Sep 7, 2018

Generate code coverage output with coverlet octokit/octokit.net#1866

Merged

3 tasks

StanleyGoldman closed this as completed Sep 10, 2018

ob-stripe mentioned this issue Oct 1, 2018

Empty results when generating coverage for Stripe.net #205

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Empty results generating report for Octokit.Net #188

Empty results generating report for Octokit.Net #188

StanleyGoldman commented Sep 7, 2018

tonerdo commented Sep 7, 2018

tonerdo commented Sep 7, 2018

StanleyGoldman commented Sep 7, 2018

StanleyGoldman commented Sep 7, 2018

tonerdo commented Sep 7, 2018

StanleyGoldman commented Sep 8, 2018

tonerdo commented Sep 9, 2018 •

edited

Loading

pjanotti commented Sep 10, 2018

StanleyGoldman commented Sep 10, 2018

tonerdo commented Sep 10, 2018

Empty results generating report for Octokit.Net #188

Empty results generating report for Octokit.Net #188

Comments

StanleyGoldman commented Sep 7, 2018

tonerdo commented Sep 7, 2018

tonerdo commented Sep 7, 2018

StanleyGoldman commented Sep 7, 2018

StanleyGoldman commented Sep 7, 2018

tonerdo commented Sep 7, 2018

StanleyGoldman commented Sep 8, 2018

tonerdo commented Sep 9, 2018 • edited Loading

pjanotti commented Sep 10, 2018

StanleyGoldman commented Sep 10, 2018

tonerdo commented Sep 10, 2018

tonerdo commented Sep 9, 2018 •

edited

Loading