Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate xpack filebeat pipeline #38420

Merged
merged 12 commits into from
Mar 20, 2024
Merged

Migrate xpack filebeat pipeline #38420

merged 12 commits into from
Mar 20, 2024

Conversation

sharbuz
Copy link
Contributor

@sharbuz sharbuz commented Mar 19, 2024 •
edited
Loading

What is the problem this PR solves?

Jenkins->Buildkite pipelines migration

Migrate x-pack filebeat pipeline

Example of the pipelines:
https://buildkite.com/elastic/beats-xpack-filebeat/builds/230

Examples of the affected pipelines:
beats-libbeat
beats-metricbeat
beats-packetbeat
beats-winlogbeat

beats-xpack-auditbeat
beats-xpack-dockerlogbeat
beats-xpack-libbeat
beats-xpack-metricbeat - there is an issue that was absent when the pipeline was migrated and it looks like the issue doesn't depend on the PR's changes
beats-xpack-packetbeat
beats-xpack-winlogbeat

Related issues

https://github.com/elastic/ingest-dev/issues/1693
https://github.com/elastic/ingest-dev/issues/2993 - related to MODULE

@sharbuz sharbuz added macOS Enable builds in the CI for darwin testing arm Enable builds in the CI for ARM testing aws Enable builds in the CI for aws cloud testing backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.12.0 Automated backport with mergify backport-v8.13.0 Automated backport with mergify labels Mar 19, 2024
@sharbuz sharbuz self-assigned this Mar 19, 2024
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Mar 19, 2024
@botelastic
Copy link

botelastic bot commented Mar 19, 2024

This pull request doesn't have a Team:<team> label.

@elasticmachine
Copy link
Collaborator

elasticmachine commented Mar 19, 2024 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Duration: 11 min 8 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@elasticmachine
Copy link
Collaborator

elasticmachine commented Mar 20, 2024 •
edited
Loading

💔 Build Failed

Failed CI Steps

History

cc @sharbuz

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @sharbuz

@sharbuz
Copy link
Contributor Author

sharbuz commented Mar 20, 2024

/test x-pack/metricbeat for aws cloud

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @sharbuz

@elasticmachine
Copy link
Collaborator

elasticmachine commented Mar 20, 2024 •
edited
Loading

💔 Build Failed

Failed CI Steps

History

cc @sharbuz

@sharbuz sharbuz requested review from dliappis and v1v March 20, 2024 10:40
@sharbuz sharbuz marked this pull request as ready for review March 20, 2024 10:40
@sharbuz sharbuz requested a review from a team as a code owner March 20, 2024 10:40
@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @sharbuz

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @sharbuz

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @sharbuz

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @sharbuz

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @sharbuz

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

History

cc @sharbuz

dliappis
dliappis approved these changes Mar 20, 2024
View reviewed changes
Copy link
Contributor

@dliappis dliappis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

.buildkite/x-pack/pipeline.xpack.filebeat.yml
IMAGE_WIN_11: "family/general-windows-11"
IMAGE_WIN_2016: "family/core-windows-2016"
IMAGE_WIN_2019: "family/core-windows-2019"
IMAGE_WIN_2022: "family/core-windows-2022"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll be switching to our custom VM images at a later stage?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to switch to our images at one time, at the end of the main part of the migration, when we have all green pipelines and then we'll compare the result. It will be easier to troubleshoot.

@sharbuz sharbuz enabled auto-merge (squash) March 20, 2024 14:17
@sharbuz
Copy link
Contributor Author

sharbuz commented Mar 20, 2024

/test

@sharbuz sharbuz disabled auto-merge March 20, 2024 14:27
@sharbuz sharbuz merged commit 22af217 into elastic:main Mar 20, 2024
43 of 47 checks passed
This was referenced Mar 20, 2024
Merged
Merged
Merged
mergify bot pushed a commit that referenced this pull request Mar 20, 2024
@sharbuz @mergify
Migrate xpack filebeat pipeline (#38420)
4465bd5 
* migrate x-pack/filebeat pipeline

* add generate_xxx.sh script

* add aws condition

* add MODULE

* change images

* fix MODULE_DIR and add nmap

* add AWS credentials

* change instance config

* test

* revert test

(cherry picked from commit 22af217)
mergify bot pushed a commit that referenced this pull request Mar 20, 2024
@sharbuz @mergify
Migrate xpack filebeat pipeline (#38420)
938bdec 
* migrate x-pack/filebeat pipeline

* add generate_xxx.sh script

* add aws condition

* add MODULE

* change images

* fix MODULE_DIR and add nmap

* add AWS credentials

* change instance config

* test

* revert test

(cherry picked from commit 22af217)
mergify bot pushed a commit that referenced this pull request Mar 20, 2024
@sharbuz @mergify
Migrate xpack filebeat pipeline (#38420)
1526bf3 
* migrate x-pack/filebeat pipeline

* add generate_xxx.sh script

* add aws condition

* add MODULE

* change images

* fix MODULE_DIR and add nmap

* add AWS credentials

* change instance config

* test

* revert test

(cherry picked from commit 22af217)
@sharbuz sharbuz deleted the migrate-xpack-filebeat-pipeline branch March 20, 2024 14:44
sharbuz added a commit that referenced this pull request Mar 20, 2024
@mergify @sharbuz
[8.13](backport #38420) Migrate xpack filebeat pipeline (#38481)
97e503d 
* Migrate xpack filebeat pipeline (#38420)

* migrate x-pack/filebeat pipeline

* add generate_xxx.sh script

* add aws condition

* add MODULE

* change images

* fix MODULE_DIR and add nmap

* add AWS credentials

* change instance config

* test

* revert test

(cherry picked from commit 22af217)

* Update pre-command

* Update common.sh

---------

Co-authored-by: sharbuz <[email protected]>
sharbuz added a commit that referenced this pull request Mar 20, 2024
@mergify @sharbuz
[8.12](backport #38420) Migrate xpack filebeat pipeline (#38480)
d57e4d2 
* Migrate xpack filebeat pipeline (#38420)

* migrate x-pack/filebeat pipeline

* add generate_xxx.sh script

* add aws condition

* add MODULE

* change images

* fix MODULE_DIR and add nmap

* add AWS credentials

* change instance config

* test

* revert test

(cherry picked from commit 22af217)

* Update pre-command

* Update common.sh

---------

Co-authored-by: sharbuz <[email protected]>
sharbuz added a commit that referenced this pull request Mar 21, 2024
@mergify @sharbuz
[7.17](backport #38420) Migrate xpack filebeat pipeline (#38479)
061c46c 
* Migrate xpack filebeat pipeline (#38420)

* migrate x-pack/filebeat pipeline

* add generate_xxx.sh script

* add aws condition

* add MODULE

* change images

* fix MODULE_DIR and add nmap

* add AWS credentials

* change instance config

* test

* revert test

(cherry picked from commit 22af217)

* Update pre-command

* Update common.sh

---------

Co-authored-by: sharbuz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dliappis dliappis dliappis approved these changes

@v1v v1v Awaiting requested review from v1v

Assignees

@sharbuz sharbuz

Labels
arm Enable builds in the CI for ARM testing aws Enable builds in the CI for aws cloud testing backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.12.0 Automated backport with mergify backport-v8.13.0 Automated backport with mergify macOS Enable builds in the CI for darwin testing needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@sharbuz @elasticmachine @dliappis