You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Our R package test coverage Actions (e.g. test-coverage.yaml) are failing due to r-lib/actions#834. We should refactor this action into a shared composite action in this repo. We should also take this opportunity to remove the Codecov integration (since they're increasingly moving to a paid model). Let's replace it with a simple coverage badge with a check that fails if coverage is reduced.
The text was updated successfully, but these errors were encountered:
@dfsnow See some discussion here around the complexity of measuring reductions in coverage: https://community.codecov.com/t/a-decrease-in-the-coverage-percentage-is-not-a-reliable-metric/2621 Do we want to handle this naively (i.e. fail if global absolute coverage decreases) and see whether we run into the problems discussed in the thread RE: deleted code, or should we start with a more sophisticated design off the bat? I like the idea of measuring the coverage of new code and/or measuring a change in the number of lines not covered, but I don't see an easy way to implement either of those approaches with covr::package_coverage.
@jeancochrane Let's go ahead with counting lines. covr::package_coverage spits out an object that (I think) you can feed to covr::tally_coverage to get line counts. This feels like a good jr project.
Sounds good @dfsnow! Unassigning myself and putting it back On Deck. Here's some more context for whoever picks this up next:
By "a simple coverage badge with a check that fails if coverage is reduced", we mean that we should remove the codecov badge from READMEs with a test-coverage workflow, and instead use only the test-coverage badge (see the README for lightsnip for an example of a project that currently displays both badges). Since the test-coverage badge displays the status of the most recent run of the test-coverage GitHub workflow on the main branch, we'll need to refactor that logic when we factor it out into a composite action so that it executes the following steps:
Pulls the number of lines not covered in the previous run on the main branch from an AWS S3 bucket
Counts the number of lines that are not covered in the current version of the code (main branch or PR)
If the number of lines covered has increased: Fails the workflow
If the number of lines covered has not increased, and the workflow is running on the main branch: Pushes the new number of lines not covered to the remote storage location in the AWS S3 bucket
If someone picks this up who is not me, let me know when you get started and I can help get the AWS S3 infra up and running.
Our R package test coverage Actions (e.g. test-coverage.yaml) are failing due to r-lib/actions#834. We should refactor this action into a shared composite action in this repo. We should also take this opportunity to remove the Codecov integration (since they're increasingly moving to a paid model). Let's replace it with a simple coverage badge with a check that fails if coverage is reduced.
The text was updated successfully, but these errors were encountered: