Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stack test --coverage shouldn't include tests in reports #634

Closed
mboes opened this issue Jul 20, 2015 · 4 comments
Closed

stack test --coverage shouldn't include tests in reports #634

mboes opened this issue Jul 20, 2015 · 4 comments
Assignees
Milestone

Comments

@mboes
Copy link
Contributor

mboes commented Jul 20, 2015

The intention of stack test --coverage is to provide a report about test coverage, i.e. how good are tests at exercising all of the library/program. In this light, we don't care about whether tests themselves have good coverage. Typically tests have very good coverage, but programs very bad coverage. So including the tests in the results biases the overall coverage report towards being too optimistic.

Perhaps adding/removing tests from the coverage report should be the matter of a flag.

@snoyberg snoyberg added this to the 0.3.0.0 milestone Jul 20, 2015
@mgsloan
Copy link
Contributor

mgsloan commented Jul 23, 2015

Another aspect of this is that currently coverage applies to all of the packages at once. This is pretty cool, but probably not what you want as a default. With larger projects it also seems to be prone to triggering some ghc / hpc bugs (#607).

Thankfully, hpc has an --include flag which allows you to restrict to particular packages / modules, which looks like it will do the trick. Unfortunately my initial experiments with it haven't been fruitful - --include=PKG doesn't work whereas --include=PKG:MOD does. These issues don't cause errors or warning messages, though - instead you just get a coverage report of 0/0. I'll look into this more, later.

@mgsloan
Copy link
Contributor

mgsloan commented Jul 23, 2015

Got it, the package filter needs to look like --include=PKG-X-Y-Z:

@mgsloan
Copy link
Contributor

mgsloan commented Jul 23, 2015

@mboes How's this look?

@mgsloan mgsloan assigned mboes and unassigned mgsloan Jul 23, 2015
mgsloan added a commit that referenced this issue Jul 29, 2015
Restrict HPC reports to just the library the TestSuite came from #634
@mboes
Copy link
Contributor Author

mboes commented Jul 29, 2015

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants