Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

No code coverage reports are uploaded if even one matrix job fails #2088

Closed
shahzadlone opened this issue Nov 28, 2023 · 0 comments · Fixed by #2089
Closed

No code coverage reports are uploaded if even one matrix job fails #2088

shahzadlone opened this issue Nov 28, 2023 · 0 comments · Fixed by #2089
Assignees
@shahzadlone
Labels
area/testing Related to any test or testing suite ci/build This is issue is about the build or CI system, and the administration of it. code quality Related to improving code quality
Milestone
DefraDB v0.9

Comments

@shahzadlone
Copy link
Member

shahzadlone commented Nov 28, 2023 •
edited
Loading

Since we disabled fail-fast, it's now more likely to have majority of jobs pass, even if a job fails.

So we need to enable uploading whatever coverage reports we have so far.

For example:
Currently windows build is flaky, and on a windows build failure it doesn't upload any code coverage reports to codecov.
Run example: https://github.com/sourcenetwork/defradb/actions/runs/7016532008/job/19088626645
@shahzadlone shahzadlone added area/testing Related to any test or testing suite ci/build This is issue is about the build or CI system, and the administration of it. code quality Related to improving code quality labels Nov 28, 2023
@shahzadlone shahzadlone added this to the DefraDB v0.9 milestone Nov 28, 2023
@shahzadlone shahzadlone self-assigned this Nov 28, 2023
@shahzadlone shahzadlone mentioned this issue Nov 28, 2023
Merged
shahzadlone added a commit that referenced this issue Nov 28, 2023
@shahzadlone
ci(i): Upload code coverage even if some matrix jobs fail (#2089)
16b66ea 
## Relevant issue(s)

Resolves #2088 

## Description
- Enable uploading whatever coverage reports we have (even if a matrix
job failed).
- Don't upload coverage reports if manually canceled the workflow/jobs.
- Fix coverage reports that were overwritten because of OS name clashing
(was uploading only 18 artifacts instead of 20).
- Increase retention period to 7 days, for the coverage reports.


For example:
Currently windows build is flaky, and on a windows build failure it
doesn't upload any code coverage reports to codecov.
Before this PR the coverage upload job skips:
https://github.com/sourcenetwork/defradb/actions/runs/7016532008/job/19088626645

## Testing

### One Fail so 19 artifacts + Upload successful:
Here is a test run to see this working in action:
https://github.com/sourcenetwork/defradb/actions/runs/7018970859/attempts/1?pr=2089pr%3D2089
Note: even with 1 failure the build still uploads the report and the
total artifact count is 19 (-1 due to windows failure).


### Passed with 20 artifacts + Upload successful:
https://github.com/sourcenetwork/defradb/actions/runs/7018970859
shahzadlone added a commit to shahzadlone/defradb that referenced this issue Jan 22, 2024
@shahzadlone
ci(i): Upload code coverage even if some matrix jobs fail (sourcenetw…
36ecf8d 
…ork#2089)

## Relevant issue(s)

Resolves sourcenetwork#2088 

## Description
- Enable uploading whatever coverage reports we have (even if a matrix
job failed).
- Don't upload coverage reports if manually canceled the workflow/jobs.
- Fix coverage reports that were overwritten because of OS name clashing
(was uploading only 18 artifacts instead of 20).
- Increase retention period to 7 days, for the coverage reports.


For example:
Currently windows build is flaky, and on a windows build failure it
doesn't upload any code coverage reports to codecov.
Before this PR the coverage upload job skips:
https://github.com/sourcenetwork/defradb/actions/runs/7016532008/job/19088626645

## Testing

### One Fail so 19 artifacts + Upload successful:
Here is a test run to see this working in action:
https://github.com/sourcenetwork/defradb/actions/runs/7018970859/attempts/1?pr=2089pr%3D2089
Note: even with 1 failure the build still uploads the report and the
total artifact count is 19 (-1 due to windows failure).


### Passed with 20 artifacts + Upload successful:
https://github.com/sourcenetwork/defradb/actions/runs/7018970859
shahzadlone added a commit to shahzadlone/defradb that referenced this issue Feb 23, 2024
@shahzadlone
ci(i): Upload code coverage even if some matrix jobs fail (sourcenetw…
64e2ee4 
…ork#2089)

## Relevant issue(s)

Resolves sourcenetwork#2088 

## Description
- Enable uploading whatever coverage reports we have (even if a matrix
job failed).
- Don't upload coverage reports if manually canceled the workflow/jobs.
- Fix coverage reports that were overwritten because of OS name clashing
(was uploading only 18 artifacts instead of 20).
- Increase retention period to 7 days, for the coverage reports.


For example:
Currently windows build is flaky, and on a windows build failure it
doesn't upload any code coverage reports to codecov.
Before this PR the coverage upload job skips:
https://github.com/sourcenetwork/defradb/actions/runs/7016532008/job/19088626645

## Testing

### One Fail so 19 artifacts + Upload successful:
Here is a test run to see this working in action:
https://github.com/sourcenetwork/defradb/actions/runs/7018970859/attempts/1?pr=2089pr%3D2089
Note: even with 1 failure the build still uploads the report and the
total artifact count is 19 (-1 due to windows failure).


### Passed with 20 artifacts + Upload successful:
https://github.com/sourcenetwork/defradb/actions/runs/7018970859
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@shahzadlone shahzadlone

Labels
area/testing Related to any test or testing suite ci/build This is issue is about the build or CI system, and the administration of it. code quality Related to improving code quality
Projects
None yet
Milestone
DefraDB v0.9
Development

Successfully merging a pull request may close this issue.

ci(i): Upload code coverage even if some matrix jobs fail shahzadlone/defradb
1 participant
@shahzadlone