-
Notifications
You must be signed in to change notification settings - Fork 179
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci(js): gate expensive e2e matrices and builds on unit test pass #9908
Conversation
Codecov Report
@@ Coverage Diff @@
## edge #9908 +/- ##
==========================================
+ Coverage 74.99% 75.02% +0.02%
==========================================
Files 2021 2023 +2
Lines 53566 53618 +52
Branches 5221 5231 +10
==========================================
+ Hits 40174 40228 +54
+ Misses 12353 12344 -9
- Partials 1039 1046 +7
Flags with carried forward coverage won't be shown. Click here to find out more.
|
@Opentrons/app-and-uis @mcous Do you think it makes sense to make sandbox builds and full app builds actually opt-in with comment handler workflows? How many FE prs do we actually use the CI artifacts from vs local builds? |
It's an interesting question. I think the main benefit of the sandbox builds isn't the sandbox publish, but the fact that CI is veryfying the change set not only passes tests, but doesn't break the build. The publish to sandbox is almost incidental, but also the lightest-weight part of the check |
I don't know about these workflows specifically, but those filters might have actually been doing something despite looking redundant. See this note in another workflow: opentrons/.github/workflows/docs-build.yaml Lines 17 to 22 in 1f34210
Added in this PR. Also note the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes all sound reasonable to me ⚖️
Overview
After the huge CI runner backlog today caused by a performance degradation in Apple's notarization service, I have a few proposals for reducing our CI load. This PR is one of them!
This PR gates more expensive tests in the front-end CI workflows behind passing the initial unit test job. This has a couple downsides:
For (1), I'd posit that we're ignoring the vast majority of these builds, anyway, especially if the unit tests aren't passing. If you are relying on the ability of sandbox builds to build with a failing test suite, you... shouldn't be doing that.
For (2), I don't think our workflow time is the thing that's getting us, it's machine usage. This strategy will tie up fewer machines.
Overall, I think the upside for all developers (not just FE devs) in fewer CI machines getting tied up greatly outweighs the downsides of this approach.
Changelog
Remove some redundantUpdate branch filters tobranch: "*"
filters that weren't doing anything**
to accept branches including slashesReview requests
👍 if you think these changes make sense
Risk assessment
Low. Increasing the strictness of our CI jobs