-
Notifications
You must be signed in to change notification settings - Fork 37
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Split tests into groups for parallel execution in CI/CD runners #417
Conversation
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## develop #417 +/- ##
===========================================
+ Coverage 89.60% 89.65% +0.05%
===========================================
Files 59 59
Lines 3617 3617
===========================================
+ Hits 3241 3243 +2
+ Misses 376 374 -2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the divide-and-conquer approach. Nevertheless, the probability of hitting the pre-existing issue with Codecov is multiplied by a factor of ~3, thus I'd suggest we decide whether we are going to mark as passed the action even if codecov failed and, if that's the case, we do it as part of this PR.
Looks like all the tests ran in between 9 to 18 minutes, btw |
Note: subdirectories |
Per @al-rigazzi comment about codecov... Do we want to do a minor rearrange of the build to avoid 3x the uploads? Perhaps we could:
The key difference in my head is to pull all artifacts instead of look for ./coverage.xml (and to figure out how to make the step depend on each matrix-based step). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Approving & assuming you'll want to split out anything about rearranging to upload codecov results fewer times into a new ticket.
runs-on: ${{ matrix.os }} | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
subset: [backends, slow_tests, group_a, group_b] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Other than the opaque group names, this is simple & effective. LGTM as the first step toward test domination!
Split tests into groups for parallel execution in CI/CD runners
Remove some unused import statements