ci: disable stages if only docs changes or .github #2877
Conversation
v1v
added
Team:Automation
💔 Build Failed
Expand to view the summary
Build stats
Test stats 🧪
Steps errors
Expand to view the steps failures
|
.ci/Jenkinsfile
Outdated
| @@ -41,9 +41,16 @@ pipeline { | |||
| deleteDir() | |||
| gitCheckout(basedir: "${BASE_DIR}") | |||
| stashV2(name: 'source', bucket: "${JOB_GCS_BUCKET}", credentialsId: "${JOB_GCS_CREDENTIALS}") | |||
| dir("${BASE_DIR}"){ | |||
| // Skip all the stages except check Go sources for changeset's with asciidoc, md or .github changes only | |||
| setEnvVar('ONLY_DOCS', isGitRegionMatch(patterns: [ '(.*\\.(asciidoc|md)|^\\.github/.*)' ], shouldMatchAll: true).toString()) | |||
There was a problem hiding this comment.
We want to run mage check if .github/CODEOWNERS change (see #2768). If this is a problem we could extract this to a different target that is always run, though mage check is fast now.
| when { | ||
| expression { return env.ONLY_DOCS == "false" } | ||
| } |
There was a problem hiding this comment.
Packages have generated content in their README.md files, elastic-package lint or elastic-package check should be executed if doc files are changed, to ensure that the generated content hasn't been unexpectedly modified.
Maybe instead of having the condition here for the whole stage, it could be after the sh that runs elastic-package check below.
There was a problem hiding this comment.
Hi, did we analyze the current costs of running? What we have right now is spawning nodes only for the affected integrations, which I would consider as a required minimum. If we want to optimize it further, I'd like to talk and identify first the most expensive components, instead of sneaking conditions.
|
|
We have found this particular behaviour in addition to #2878 and also the missing tear down in elastic/elastic-package#701 As a consequence, I looked at the pipeline and raised two different PRs to reduce the number of builds in the CI for changes that potentially are not needed. |
|
Maybe we should run different pipelines for different elements? Like "docs pipeline", "integrations pipeline", etc. I agree we don't need to spawn a hundred nodes to check docs, but the last time we changed docs was 9 days ago, 2 months ago, 12 months ago. |
|
To clarify, I didn't use any data in terms of justifying this change is needed, but found out that for some particular PRs won't need to run all the stages as those stages do not test anything for those changes. So, filtering what to run and when could help regardless if it's often changed or not. Even if it's not, there is a chance that accidentally someone could create a few PRs and then cope with all the CI capacity for instance :/ |
No strong opinion, even by doing so, it's required to create the filtering. So adding the filtering at the stage level reduces the number of GitHub api calls. |
What does this PR do?
Faster builds for changes which are not verified/validated by the build system
Why
Smart builds reduce the cost (money and time) for commits which don't need to be validated
Therefore, #2874 could be a good candidate to be done in a few minutes rather than dozens of minutes.