Coverage for tests

[MLopez-Ibanez]

It would be useful to report coverage for files under tests:

1. The tests directory may contain code that is not a test itself but it is supposed to be used by tests. If that code is not used, it can probably be removed.
2. Knowing which tests are actually run (and if a test stopped running for some reason) is useful.
3. pytest --covr does report coverage for tests, which allows setting different codecov components/flags with different status criteria for R and tests. (For example, we may require that coverage for tests cannot decrease of the CI fails)

The option covr.record_tests does a lot more than just recording coverage for tests.

[jimhester]

I can see how this may be useful.

Unfortunately due to the way that covr works internally this isn't really easily done so I don't feel the added value would justify the time required to implement and maintain this.

[MLopez-Ibanez]

Unfortunately due to the way that covr works internally this isn't really easily done so I don't feel the added value would justify the time required to implement and maintain this.

I see. I looked at the code and it seems testInstalledPackage() does not have any appropriate entry points where to call trace_environment().

Is this something that would require support from the tools package or the R parser? I could open a feature request in R's bugzilla if so.

[jimhester]

The problem is more related to how covr tracks coverage. In order to track coverage it needs to instrument the code and add tracing calls, this is done by instrumenting functions in the package namespace when the package is loaded. Code within test files is not included in the package namespace.

However you could try to cobble something together yourself by using file_coverage() to instrument the files within tests and combining the results with package_coverage().

[MLopez-Ibanez]

However you could try to cobble something together yourself by using file_coverage() to instrument the files within tests and combining the results with package_coverage().

I am happy to give this a try but the first obstacle is that source_files will execute the tests immediately. Is there a way to instrument the files under tests/testthat without executing them?