Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: maybe remove Codecov? #8185

Closed
2 tasks
mcous opened this issue Jul 30, 2021 · 2 comments · Fixed by #8190
Closed
2 tasks

RFC: maybe remove Codecov? #8185

mcous opened this issue Jul 30, 2021 · 2 comments · Fixed by #8190
Labels
infrastructure ops/ CI/ repo etc RFC Software proposal ("request for comment")

Comments

@mcous
Copy link
Contributor

mcous commented Jul 30, 2021

Overview

We started using Codecov to collect coverage statistics back in the heady days of #313. Those were simpler times, when we had an api project and an app project. Our previous coverage statistics provider, Coveralls, didn't support multiple languages at the time, hence the switch.

To say our Codecov coverage collection hasn't scaled gracefully would be putting it mildly. I think we have all collectively experienced the following:

  • Nonsensical coverage comments on most PRs, including coverage changes for untouched files
  • That uh, huge security incident
  • Overall coverage statistics that don't even include all our projects, which was the original goal of codecov

Could we fix it? Maybe. Does anyone feel like it? I don't. Should we get rid of Codecov? Might be nice.

Acceptance Criteria

  • Remove codecov config and deactivate our repo in the Codecov UI
  • Remove (currently wildly inaccurate) coverage badge from README.md
@mcous mcous added RFC Software proposal ("request for comment") infrastructure ops/ CI/ repo etc labels Jul 30, 2021
@SyntaxColoring
Copy link
Contributor

In theory, coverage comments on PRs sound really nice. If I add some new code, it's great to have a way of seeing if I forgot to cover some obscure execution path in that code with a test.

In practice, it's hard to trust Codecov's nonsensical coverage comments. 🙁

Would things be better or worse if we applied Codecov to individual projects, instead of the whole repo? Is that even possible? I wonder if our fancy CI that avoids unnecessarily rebuilding unrelated subprojects is messing with our coverage uploads to Codecov, or something.

@mcous
Copy link
Contributor Author

mcous commented Jul 30, 2021

Codecov flags might be a potential solution here!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infrastructure ops/ CI/ repo etc RFC Software proposal ("request for comment")
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants