Skip to content

Commit

Permalink
fix(ci): Temporarily allow ember-release tests to fail in CI (#4227)
Browse files Browse the repository at this point in the history
Right now, tests against the current release of ember are failing, because ember 4.0 brought changes our SDK isn't equipped to handle without breaking changes of its own. (See #4179.) Until we make those changes, we know that test scenario is going to fail.

Meanwhile, GH is inconsistent in terms of what it considers success or failure in CI. GHA has a setting, `continue-on-error`, both for jobs[1] and for steps[2], which is essentially an xfail for a job within a workflow run (in other words, that job failing doesn't make the whole workflow run fail) and for a step within a job, respectively. This (in theory) allows processes which depend on the workflow or job succeeding to keep humming right along without the failure being silently swallowed.

In our case, we have this set on the job running ember tests, so that if they fail, the entire workflow shouldn't. Indeed, if you look in the Actions tab, it shows up as green, in spite of the ember failures. But in terms of the little X or check next to commits/branches, and in terms of what it reports to craft, it shows up as red. This means that if anything else breaks, we won't know, since we'll already be expecting CI to be broken. It also prevents releases.

This xfails the `ember-release` suite so that we can proceed with the rest of the repo as normal. We have the ticket linked above, and it's in our plan for the next major release (if it's not fixed before then), so we won't lose track of getting them fixed, even though we won't have the red X staring us in the face.

[1] https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#jobsjob_idcontinue-on-error
[2] https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#jobsjob_idstepscontinue-on-error
  • Loading branch information
lobsterkatie authored Dec 3, 2021
1 parent 111e1ff commit 7e5578e
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions packages/ember/config/ember-try.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ module.exports = async function() {
'ember-source': await getChannelURL('release'),
},
},
allowedToFail: true,
},
];

Expand Down

0 comments on commit 7e5578e

Please sign in to comment.