Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[7.17](backport #39506) Remove hardcoded module definitions in CI #39527

Merged
merged 2 commits into from
May 13, 2024
Merged

[7.17](backport #39506) Remove hardcoded module definitions in CI #39527

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4dba24e
Remove hardcoded module definitions in CI (#39506)
dliappis May 13, 2024
40df916
fix conflict
dliappis May 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions .buildkite/metricbeat/pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,13 +50,10 @@ steps:
key: "mandatory-int-test"
command: |
set -euo pipefail

# TODO: uncomment all lines below (that set MODULE and install kind) when https://github.com/elastic/beats/issues/38874 has been resolved.
# TODO: Note that this problem exists ONLY on 7.17 branch
# # defines the MODULE env var based on what's changed in a PR
# source .buildkite/scripts/changesets.sh
# defineModuleFromTheChangeSet metricbeat
# echo "~~~ Will run tests with env var MODULE=$$MODULE"
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet metricbeat
echo "~~~ Running tests"

# TODO move this section to base image / pre-command hook
# echo "~~~ Installing kind"
Expand Down Expand Up @@ -90,7 +87,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet metricbeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"

# TODO move this section to base image / pre-command hook
echo "~~~ Installing kind"
Expand Down
16 changes: 4 additions & 12 deletions .buildkite/scripts/changesets.psm1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,17 +56,9 @@ function DefineModuleFromTheChangeSet($projectPath) {
}
}

# TODO: remove this conditional when issue https://github.com/elastic/ingest-dev/issues/2993 gets resolved
if(!$changedModules) {
if($Env:BUILDKITE_PIPELINE_SLUG -eq 'beats-xpack-metricbeat') {
$Env:MODULE = "aws"
}
else {
$Env:MODULE = "kubernetes"
}
}
else {
# TODO: once https://github.com/elastic/ingest-dev/issues/2993 gets resolved, this should be the only thing we export
$Env:MODULE = $changedModules
if ($changedModules) {
$env:MODULE = $changedModules
Write-Output "~~~ Set env var MODULE to [$env:MODULE]"
Write-Output "~~~ Resuming commands"
}
}
13 changes: 5 additions & 8 deletions .buildkite/scripts/changesets.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,13 +68,10 @@ defineModuleFromTheChangeSet() {
fi
done

if [[ -z "$changed_modules" ]]; then # TODO: remove this conditional when issue https://github.com/elastic/ingest-dev/issues/2993 gets resolved
if [[ "$BUILDKITE_PIPELINE_SLUG" == "beats-xpack-metricbeat" ]]; then
export MODULE="aws"
else
export MODULE="kubernetes"
fi
else
export MODULE="${changed_modules}" # TODO: once https://github.com/elastic/ingest-dev/issues/2993 gets resolved, this should be the only thing we export
# export MODULE="" leads to an infinite loop https://github.com/elastic/ingest-dev/issues/2993
if [[ ! -z $changed_modules ]]; then
export MODULE="${changed_modules}"
echo "~~~ Set env var MODULE to [$MODULE]"
echo "~~~ Resuming commands"
fi
}
2 changes: 1 addition & 1 deletion .buildkite/x-pack/pipeline.xpack.auditbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/auditbeat
echo "~~~ Will run tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/auditbeat
mage update build test
retry:
Expand Down
4 changes: 1 addition & 3 deletions .buildkite/x-pack/pipeline.xpack.dockerlogbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,9 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/dockerlogbeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/dockerlogbeat
mage goIntegTest
env:
MODULE: $MODULE
retry:
automatic:
- limit: 3
Expand Down
8 changes: 4 additions & 4 deletions .buildkite/x-pack/pipeline.xpack.filebeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/filebeat
echo "~~~ Will run tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/filebeat && mage goIntegTest
retry:
automatic:
Expand All @@ -74,7 +74,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/filebeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/filebeat && mage pythonIntegTest
retry:
automatic:
Expand Down Expand Up @@ -268,7 +268,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/filebeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"

.buildkite/scripts/cloud_tests.sh
env:
Expand Down Expand Up @@ -297,7 +297,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/filebeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
.buildkite/scripts/cloud_tests.sh
env:
ASDF_TERRAFORM_VERSION: 1.0.2
Expand Down
8 changes: 4 additions & 4 deletions .buildkite/x-pack/pipeline.xpack.metricbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/metricbeat
echo "~~~ Will run tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/metricbeat && mage goIntegTest
retry:
automatic:
Expand All @@ -74,7 +74,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/metricbeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/metricbeat && mage pythonIntegTest
retry:
automatic:
Expand Down Expand Up @@ -253,7 +253,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/metricbeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"

.buildkite/scripts/cloud_tests.sh
env:
Expand Down Expand Up @@ -284,7 +284,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/metricbeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
.buildkite/scripts/cloud_tests.sh
env:
ASDF_TERRAFORM_VERSION: 1.0.2
Expand Down
4 changes: 1 addition & 3 deletions .buildkite/x-pack/pipeline.xpack.winlogbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,9 @@ steps:
command: |
Import-Module ./.buildkite/scripts/changesets.psm1
defineModuleFromTheChangeSet 'x-pack/winlogbeat'
Write-Output "~~~ Will run tests with env var MODULE=$$Env:MODULE"
Write-Output "~~~ Running tests"
Set-Location -Path x-pack/winlogbeat
mage build unitTest
env:
MODULE: $MODULE
retry:
automatic:
- limit: 3
Expand Down
Loading