replace default images #38583

sharbuz · 2024-03-25T09:12:24Z

What is the problem this PR solves?

Jenkins->Buildkite pipelines migration

Replace default images by our own images

Example of the pipelines:

Examples of the affected pipelines:

Related issues

https://github.com/elastic/ingest-dev/issues/1693

botelastic · 2024-03-25T09:12:36Z

This pull request doesn't have a Team:<team> label.

elasticmachine · 2024-03-25T09:26:15Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2024-04-05T05:45:52.567+0000
Duration: 17 min 33 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3
Skipped	0
Total	3

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages and run the E2E tests.
/beats-tester : Run the installation tests with beats-tester.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

Signed-off-by: Alexandros Sapranidis <[email protected]>

…e-images

alexsapran · 2024-04-04T10:05:38Z

.buildkite/hooks/pre-command

 GITHUB_TOKEN_VAULT_PATH="kv/ci-shared/platform-ingest/github_token"

 if [[ "$BUILDKITE_PIPELINE_SLUG" == "beats" || "$BUILDKITE_PIPELINE_SLUG" == "filebeat" || "$BUILDKITE_PIPELINE_SLUG" == "auditbeat" || "$BUILDKITE_PIPELINE_SLUG" == "heartbeat" || "$BUILDKITE_PIPELINE_SLUG" == "deploy-k8s" ]]; then
  source .buildkite/env-scripts/env.sh
+  if [[ -z "${GO_VERSION-""}" ]]; then


Why do we need to do this now? This is handled inside the VM image.

.buildkite/scripts/generate_auditbeat_pipeline.sh

alexsapran · 2024-04-04T10:30:09Z

.buildkite/scripts/generate_filebeat_pipeline.sh

How are we adding this file? Didn't Filebeat already exist?

there is the problem of conflict-solving after yesterday's merge.

.buildkite/scripts/generate_filebeat_pipeline.sh

pazone · 2024-04-04T13:57:43Z

LGTM.
This PR contains a lot of work and makes big progress.
To reviewers: Please make sure that you leave comments about only critical findings. Let's move it forward

alexsapran

Overall, it looks good to me +1. I only noticed one Windows step that had the wrong command.
I synced with @pazone to investigate the package step. If there is an issue with that step, I would like to address it before landing this massive PR.
@pazone, please approve it once you know so we can merge it.

.buildkite/scripts/generate_xpack_packetbeat_pipeline.sh

pazone · 2024-04-04T14:51:58Z

Looks like the package steps can be removed for PRs.
I propose to merge this PR and I can disable those steps later.

pazone · 2024-04-04T15:43:32Z

@sharbuz has changed the packaging script to mage package and now we don't publish artifacts on PRs. @alexsapran

* replace default images -------------------------- Signed-off-by: Alexandros Sapranidis <[email protected]> Co-authored-by: Alexandros Sapranidis <[email protected]> (cherry picked from commit 80dab50) # Conflicts: # .buildkite/auditbeat/auditbeat-pipeline.yml # .buildkite/filebeat/filebeat-pipeline.yml # .buildkite/heartbeat/heartbeat-pipeline.yml # .buildkite/hooks/pre-command # .buildkite/scripts/generate_auditbeat_pipeline.sh # .buildkite/scripts/generate_filebeat_pipeline.sh # .buildkite/scripts/packaging/package-step.sh # .buildkite/scripts/setenv.sh

* replace default images -------------------------- Signed-off-by: Alexandros Sapranidis <[email protected]> Co-authored-by: Alexandros Sapranidis <[email protected]> (cherry picked from commit 80dab50) # Conflicts: # .buildkite/auditbeat/auditbeat-pipeline.yml # .buildkite/filebeat/filebeat-pipeline.yml # .buildkite/filebeat/scripts/package-step.sh # .buildkite/heartbeat/heartbeat-pipeline.yml # .buildkite/hooks/pre-command # .buildkite/scripts/common.sh # .buildkite/scripts/setenv.sh

* replace default images -------------------------- Signed-off-by: Alexandros Sapranidis <[email protected]> Co-authored-by: Alexandros Sapranidis <[email protected]> (cherry picked from commit 80dab50) # Conflicts: # .buildkite/auditbeat/auditbeat-pipeline.yml # .buildkite/filebeat/filebeat-pipeline.yml # .buildkite/heartbeat/heartbeat-pipeline.yml # .buildkite/hooks/pre-command # .buildkite/scripts/generate_auditbeat_pipeline.sh # .buildkite/scripts/generate_filebeat_pipeline.sh # .buildkite/scripts/setenv.sh

* replace default images -------------------------- Signed-off-by: Alexandros Sapranidis <[email protected]> Co-authored-by: Alexandros Sapranidis <[email protected]>

* replace default images (#38583) * replace default images -------------------------- Signed-off-by: Alexandros Sapranidis <[email protected]> Co-authored-by: Alexandros Sapranidis <[email protected]> (cherry picked from commit 80dab50) # Conflicts: # .buildkite/auditbeat/auditbeat-pipeline.yml # .buildkite/filebeat/filebeat-pipeline.yml # .buildkite/filebeat/scripts/package-step.sh # .buildkite/heartbeat/heartbeat-pipeline.yml # .buildkite/hooks/pre-command # .buildkite/scripts/common.sh # .buildkite/scripts/setenv.sh * Update auditbeat-pipeline.yml * Update filebeat-pipeline.yml * Update package-step.sh * Update package-step.sh * Update heartbeat-pipeline.yml * Update pre-command * Update pre-command * Update pre-command * resolve conflicts * Update common.sh * Update setenv.sh * Update pre-command --------- Co-authored-by: Siarhei Harbuz <[email protected]> Co-authored-by: Siarhei Harbuz <[email protected]>

* replace default images (#38583) * replace default images -------------------------- Signed-off-by: Alexandros Sapranidis <[email protected]> Co-authored-by: Alexandros Sapranidis <[email protected]> (cherry picked from commit 80dab50) # Conflicts: # .buildkite/auditbeat/auditbeat-pipeline.yml # .buildkite/filebeat/filebeat-pipeline.yml # .buildkite/heartbeat/heartbeat-pipeline.yml # .buildkite/hooks/pre-command # .buildkite/scripts/generate_auditbeat_pipeline.sh # .buildkite/scripts/generate_filebeat_pipeline.sh # .buildkite/scripts/setenv.sh * Update auditbeat-pipeline.yml * Update filebeat-pipeline.yml * Update heartbeat-pipeline.yml * Update pre-command * Update generate_auditbeat_pipeline.sh * Update generate_filebeat_pipeline.sh * Update setenv.sh --------- Co-authored-by: Siarhei Harbuz <[email protected]> Co-authored-by: Siarhei Harbuz <[email protected]>

replace default images

4029c11

sharbuz self-assigned this Mar 25, 2024

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 25, 2024

export variables

65558fd

Siarhei Harbuz added 17 commits March 25, 2024 13:16

change libbeat pipeline

7c283cf

change metricbeat pipeline

7158bfd

change packetbeat pipeline

cdc3c5b

change winlogbeat pipeline

9488b70

change auditbeat pipeline

9289bd8

change filebeat pipeline

ada8c3d

change heartbeat pipeline

a68366c

change libbeat pipeline

af4ac56

change xpack-metricbeat pipeline

6b14d12

change xpack-osquerybeat pipeline

8a8e86d

change xpack-packetbeat pipeline

4301b18

change xpack-winlogbeat pipeline

71c4cc7

change images

88a6ebe

change windows command and 'make -c' command

f90e431

update pre-command

e08451d

update x-pack/heartbeat

2fbdae9

change windows command and add MODULE into metricbeat pipeline

bfcf87e

alexsapran and others added 7 commits April 3, 2024 14:21

Cleanup winlogbeat pipeline yaml

c83f74a

Signed-off-by: Alexandros Sapranidis <[email protected]>

Remove qa image use

e8e0963

Signed-off-by: Alexandros Sapranidis <[email protected]>

merge main and resolve conflicts

3bfc70b

Merge branch 'replace-images' of github.com:sharbuz/beats into replac…

33d945c

…e-images

merge main and resolve conflicts

4bb7858

Merge branch 'main' into replace-images

f7386e0

macos auditbeat test

5ee077f

alexsapran reviewed Apr 4, 2024

View reviewed changes

.buildkite/scripts/generate_filebeat_pipeline.sh Outdated Show resolved Hide resolved

sort pipelines' environment variables

50e6496

sharbuz requested a review from pazone April 4, 2024 14:00

change artifact_path and packaging steps

55fa3b0

alexsapran reviewed Apr 4, 2024

View reviewed changes

.buildkite/scripts/generate_xpack_packetbeat_pipeline.sh Show resolved Hide resolved

pazone approved these changes Apr 4, 2024

View reviewed changes

Siarhei Harbuz added 2 commits April 4, 2024 18:03

sugaring

f1bd614

change packaging

9ebd748

fix ununtu arm images

d54bb83

sharbuz merged commit 80dab50 into elastic:main Apr 5, 2024
123 of 125 checks passed

This was referenced Apr 5, 2024

[7.17](backport #38583) replace default images #38731

Closed

[8.12](backport #38583) replace default images #38732

Merged

mergify bot mentioned this pull request Apr 5, 2024

[8.13](backport #38583) replace default images #38733

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

replace default images #38583

replace default images #38583

sharbuz commented Mar 25, 2024

botelastic bot commented Mar 25, 2024

elasticmachine commented Mar 25, 2024 •

edited

Loading

Build stats

Test stats 🧪

alexsapran Apr 4, 2024

alexsapran Apr 4, 2024

sharbuz Apr 4, 2024

pazone commented Apr 4, 2024

alexsapran left a comment •

edited

Loading

pazone commented Apr 4, 2024

pazone commented Apr 4, 2024 •

edited

Loading

replace default images #38583

replace default images #38583

Conversation

sharbuz commented Mar 25, 2024

What is the problem this PR solves?

Related issues

botelastic bot commented Mar 25, 2024

elasticmachine commented Mar 25, 2024 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

alexsapran Apr 4, 2024

Choose a reason for hiding this comment

alexsapran Apr 4, 2024

Choose a reason for hiding this comment

sharbuz Apr 4, 2024

Choose a reason for hiding this comment

pazone commented Apr 4, 2024

alexsapran left a comment • edited Loading

Choose a reason for hiding this comment

pazone commented Apr 4, 2024

pazone commented Apr 4, 2024 • edited Loading

elasticmachine commented Mar 25, 2024 •

edited

Loading

alexsapran left a comment •

edited

Loading

pazone commented Apr 4, 2024 •

edited

Loading