Session Proposal: CI Stability - how do we get it green?

[sam-github]

Proposal

Topic of the session

CI Stability - how do we get it green?

Brainstorm approachs to getting CI green.

Daily builds always fail, and since master can't build Green in a single shot, neither can a PR. Having to re-run CI repeatedly for even a simple change to confirm it doesn't introduce a regression is burdensome, and risks people starting to ignore our CI because its false positive rate is too high.

We have improved the chance of success a bit lately by agressively marking tests as flaky, but this amounts to ignoring tests so is a problematic strategy.

• failing daily builds: https://ci.nodejs.org/job/node-daily-master/
• flaky tests: https://github.com/nodejs/node/issues?q=is%3Aopen+is%3Aissue+label%3A%22CI+%2F+flaky+test%22

Type of the session

• Collaborate
• Workshop
• Talk

Timezone of facilitator

• Eastern Time Zone, North-America

Estimated duration of the session

1 Hour

Follow-up / Set-up sessions (if any)

N/A

Level

• Beginner
• Intermediate
• Advanced

Pre-requisite knowledge

Anyone in the Build WG or TSC, anyone who is involved in CI or Test fixing, triage of flaky tests, and any collaborators who are contributing frequently enough to be blocked by the frequent irrelevant build failures.

Describe the session

Session facilitator(s), Github handle(s) and timezone(s)

@sam-github, and @AshCripps (build-wg) and @Trott (active contributor to CI stability) might be interested in helping facilitate?

Additional context (optional)

---

This section is for contributors handling Collaborator Summit logistics

Track

• New Contributor Orientation Track (see New Contributor Orientation Track #251)
• Regular Track
• Cross-Project Track

Triage

• Track chosen
• Scheduled

[jorydotcom]

Hi @sam-github 👋 would you say this session is open for folks working on the same challenges in other projects, or is mostly Node specific?

[sam-github]

I think its pretty node.js specific, I'm not aware of any other opensource-in-javascript projects with ci of the scale of Node.js, is there one? I wouldn't want the conversation to be diluted by "XXX is working great for my project, it supports all 3 of our platforms, why doesn't node.js just use it for everything?". Much of our problems are related to the scale. But what do other people think?

[jorydotcom]

AMP and Electron are the other biggie scale projects, but valid point that the CI specifics probably vary too much for this to be relevant. Thanks for helping me understand!

[sam-github]

OK, I know a bit about Electron through @jkleinsc's work on the build. I think this session should be Node.js specific as to the problems to solve, though attendance by people with experience in other projects who want to help Node.js would be very welcome.

[jorydotcom]

Notes from the session room

@sam-github do you recall if this session was recorded?

[Trott]

Sam couldn't make it. I don't think we recorded it, did we @AshCripps?

[AshCripps]

umm I think @mhdawson might have got the recording sorted

[mhdawson]

Not for this one as I was not in the session.

[WaleedAshraf]

Closing this issue as no further action is needed.