Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(codecov): fix codecov / jest coverage misconfigurations #8190

Merged
merged 3 commits into from
Aug 4, 2021

Conversation

mcous
Copy link
Contributor

@mcous mcous commented Aug 2, 2021

Overview

Upon some investigation for #8185, it turns out that the port from TravisCI to incremental builds in GitHub Actions never properly configured code coverage reporting for all of our JS projects, which explains our blatantly wrong coverage percentage in codecov.

This PR fixes coverage uploads and sets up per-project Codecov flags to see if that makes a meaningful improvement to coverage comments in PRs.

Closes #4985, closes #8185

Changelog

  • Ensure unit-test CI workflows actually upload coverage reports
  • Add JS unit tests to shared-data CI
  • Upgrade to codecov-action v2
  • Configure Codecov with per-project flags and carry-forward flags for incremental monorepo build support

Review requests

  • Give the config changes a look over and see if they make sense
  • If we merge this, should we nuke our Codecov data and start fresh?
    • I think yes, the historical data is so messed up that I can't see any value in keeping it

Risk assessment

N/A, CI config

@codecov
Copy link

codecov bot commented Aug 2, 2021

Codecov Report

Merging #8190 (b8a3bb8) into edge (65d64ae) will decrease coverage by 12.73%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##             edge    #8190       +/-   ##
===========================================
- Coverage   87.44%   74.71%   -12.74%     
===========================================
  Files         444     1650     +1206     
  Lines       22817    44112    +21295     
  Branches        0     4408     +4408     
===========================================
+ Hits        19952    32957    +13005     
- Misses       2865    10411     +7546     
- Partials        0      744      +744     
Flag Coverage Δ
api 85.68% <ø> (?)
app 70.51% <ø> (?)
components 48.76% <ø> (?)
labware-library 50.22% <ø> (?)
protocol-designer 44.15% <ø> (?)
robot-server 93.42% <ø> (?)
shared-data 84.14% <ø> (?)
step-generation 90.33% <ø> (?)
update-server 74.12% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...obot-server/robot_server/sessions/action_models.py 100.00% <0.00%> (ø)
...ing/g_code_functionality_defs/tempdeck/__init__.py
...cycler/edit_pid_params_g_code_functionality_def.py
opentrons/protocol_engine/clients/transports.py
opentrons/commands/util.py
opentrons/drivers/rpi_drivers/interfaces.py
opentrons/file_runner/create_file_runner.py
...mocycler/set_ramp_rate_g_code_functionality_def.py
opentrons/protocol_engine/commands/dispense.py
...ie/write_instrument_id_g_code_functionality_def.py
... and 1816 more

@mcous mcous added infrastructure ops/ CI/ repo etc JS Tech Debt python General Python tickets and PRs; e.g. tech debt or Python guild activities labels Aug 2, 2021
@mcous mcous marked this pull request as ready for review August 2, 2021 21:24
@mcous mcous requested review from a team as code owners August 2, 2021 21:24
@mcous mcous requested review from IanLondon and removed request for a team August 2, 2021 21:24
@@ -1,14 +1,6 @@
fixes:
- 'api/src/opentrons/::opentrons/'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was present to preserve historical coverage data. Given that our coverage history is fairly messed up at this point, I figured we could remove it for a more straightforward experience.

With this line in place, the codecov UI presents an opentrons folder instead of an api folder, which is confusing

ignore:
- 'audio'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These don't exist anymore

- '**/node_modules'
- 'protocol-library-kludge'
- 'shared-data/deck/definitions'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are redundant with the JSON ignore line below

.github/workflows/app-test-build-deploy.yaml Outdated Show resolved Hide resolved
with:
file: coverage.xml
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was just wrong; jest/Istanbul writes to coverage/lcov.info by default

Copy link
Contributor

@SyntaxColoring SyntaxColoring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess there's not really any way to be sure other than testing this in "production," but this looks sensible to me!

Edit:

  • If we merge this, should we nuke our Codecov data and start fresh?
  • I think yes, the historical data is so messed up that I can't see any value in keeping it

Agreed.

shared-data/python/Pipfile Show resolved Hide resolved
.github/workflows/app-test-build-deploy.yaml Show resolved Hide resolved
.github/workflows/notify-server-lint-test.yaml Outdated Show resolved Hide resolved
jest.config.js Show resolved Hide resolved
@mcous mcous merged commit d515cc3 into edge Aug 4, 2021
@mcous mcous deleted the ci_codecov-tags branch August 4, 2021 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infrastructure ops/ CI/ repo etc JS Tech Debt python General Python tickets and PRs; e.g. tech debt or Python guild activities
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RFC: maybe remove Codecov? Revisit Codecov tooling
2 participants