Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Buildkite] Skip install package command in serverless builds for some packages #9686

Merged
merged 3 commits into from
Apr 25, 2024
Merged

[Buildkite] Skip install package command in serverless builds for some packages #9686

merged 3 commits into from
Apr 25, 2024

Conversation

mrodm
Copy link
Contributor

@mrodm mrodm commented Apr 23, 2024

Proposed commit message

Skip "elastic-package install" command in Serverless builds. "elastic-package test asset" already performs the installation of the package too.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

@mrodm mrodm self-assigned this Apr 23, 2024
@elasticmachine
Copy link

elasticmachine commented Apr 23, 2024 •
edited
Loading

🚀 Benchmarks report

Package system 👍(1) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
auth 2890.17 1949.32 -940.85 (-32.55%) 💔
security 2439.02 1960.78 -478.24 (-19.61%) 💔

To see the full report comment with /test benchmark fullreport

jsoriano
jsoriano reviewed Apr 24, 2024
View reviewed changes
.buildkite/scripts/common.sh
# "elastic-paackage test asset" command already installs the package via upload API
# as it does "elastic-package install" command
echo "--- [${package}] test installation"
if ! install_package "${package}" ; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we keep this test for some package in serverless? I would like to keep testing this command.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I've added the logic to just skip one package for now here 01e0cb3

@mrodm mrodm marked this pull request as ready for review April 24, 2024 08:43
@mrodm mrodm requested a review from a team as a code owner April 24, 2024 08:43
@mrodm mrodm requested a review from jsoriano April 24, 2024 08:43
kpollich
kpollich approved these changes Apr 24, 2024
View reviewed changes
.buildkite/scripts/common.sh
Comment on lines +764 to +768
if [[ "$package" == "security_detection_engine" ]]; then
return 0
fi

return 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine-ish for now as this package is an outlier in terms of size and impact on memory usage. We'll have to monitor whether the duplicate installations for all other packages still impact memory enough to cause an occasional OOM on QA projects.

@jsoriano
Copy link
Member

/test

@elasticmachine
Copy link

elasticmachine commented Apr 24, 2024 •
edited
Loading

⏳ Build in-progress, with failures

Failed CI Steps

History

cc @mrodm

@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@mrodm mrodm changed the title Skip install package command in serverless builds [Buildkite] Skip install package command in serverless builds for some packages Apr 25, 2024
@mrodm mrodm merged commit d13e474 into elastic:main Apr 25, 2024
5 checks passed
@mrodm mrodm deleted the remove_install_step_serverless branch April 25, 2024 09:55
gizas pushed a commit that referenced this pull request Apr 26, 2024
@mrodm @gizas
[Buildkite] Skip install package command in serverless builds for som…
1e7292b 
…e packages (#9686)
milan-elastic added a commit to milan-elastic/integrations that referenced this pull request May 1, 2024
@milan-elastic
Squashed commit of the following:
8cf8f03 
commit e2a688fbb1c8712ba0cad243713146867ac2f986
Author: milan-elastic <[email protected]>
Date:   Wed May 1 15:43:52 2024 +0530

    Squashed commit of the following:

    commit a17de73aa84608f67a1baca4c094819b562e42e0
    Author: milan-elastic <“[email protected]”>
    Date:   Wed May 1 15:29:41 2024 +0530

        Squashed commit of the following:

        commit fccdb1f83f0048b07df6ee82fbd91ca432c799b9
        Author: milan-elastic <[email protected]>
        Date:   Wed May 1 14:58:41 2024 +0530

            add global filter on dashboard level for hadoop

        commit 686e49be78dc980b2f12d365580cb800fd7cf330
        Merge: 024d864b4 01201a7
        Author: “milan-elastic” <“[email protected]”>
        Date:   Wed May 1 11:38:59 2024 +0530

            Merge branch 'main' of github.com:milan-elastic/integrations into mongodb-atlas-database-logs

        commit 01201a7
        Author: Eric Forte <[email protected]>
        Date:   Tue Apr 30 10:46:55 2024 -0400

            [Security Rules] Update security rules package to v8.13.5 (elastic#9762)

            * [Security Rules] Update security rules package to v8.13.5

            * Add changelog entry for 8.13.5

            ---------

            Co-authored-by: protectionsmachine <[email protected]>

        commit c9d1f1b
        Author: Eric Forte <[email protected]>
        Date:   Tue Apr 30 09:30:30 2024 -0400

            [Security Rules] Update security rules package to v8.13.5-beta.1 (elastic#9758)

            * [Security Rules] Update security rules package to v8.13.5-beta.1

            * Add changelog entry for 8.13.5-beta.1

            ---------

            Co-authored-by: protectionsmachine <[email protected]>

        commit a79f813
        Author: Tetiana Kravchenko <[email protected]>
        Date:   Tue Apr 30 11:32:37 2024 +0200

            [kubernetes] Remove deprecated fields, add missing status.last_terminated_reason metric (elastic#9736)

            * remove deprecated fields

            Signed-off-by: Tetiana Kravchenko <[email protected]>

            * Update changelog.yml

            * add missing metric: last_terminated_reason; update description of the status.reason field

            Signed-off-by: Tetiana Kravchenko <[email protected]>

            ---------

            Signed-off-by: Tetiana Kravchenko <[email protected]>

        commit b1627a3
        Author: ShourieG <[email protected]>
        Date:   Tue Apr 30 13:03:29 2024 +0530

            [integrations][http_endpoint] - Converted HTTP Endpoint Integration to input type (elastic#9732)

            * converted http_endpoint to input package type

            * updated changelog

            * updated original event in sample event

        commit 3a9b508
        Author: Lalit Satapathy <[email protected]>
        Date:   Tue Apr 30 11:49:09 2024 +0530

            Remove separate codeowners for system package kibana paths. (elastic#9731)

        commit c90e817
        Author: Krishna Chaitanya Reddy Burri <[email protected]>
        Date:   Tue Apr 30 11:32:17 2024 +0530

            [Crowdstrike,Azure] Fix flaky tests with ECS fields (elastic#9738)

            * Fix flaky pipeline tests.

               * `azure.graphactivitylogs`: Add missing ECS field definitions.

               * `crowdstrike.falcon`: Update `geoip` processor to `destination` instead of `source`.

        commit ace8fb4
        Author: Aliabbas Attarwala <[email protected]>
        Date:   Mon Apr 29 16:37:23 2024 +0530

            [O11y][AWS] Rally benchmark `aws.cloudtrail` (elastic#9448)

        commit d4e4aa4
        Author: niraj-elastic <[email protected]>
        Date:   Mon Apr 29 14:45:46 2024 +0530

            [Apache] Update grok pattern for accepting user-identity (elastic#9632)

            * update grok pattern

            * update changelog

            * address review comments

            * address review comments

            Co-authored-by: muthu-mps <[email protected]>

            * address review comments

            * address review comment

            ---------

            Co-authored-by: muthu-mps <[email protected]>

        commit dce5699
        Author: Mario Rodriguez Molins <[email protected]>
        Date:   Mon Apr 29 10:33:19 2024 +0200

            Enable publishing packages from integrations-publish pipeline (elastic#9712)

            Enable publishing packages from integrations-publish pipeline,
            and remove corresponding step from the main pipeline.

        commit c7bc530
        Author: Chema Martínez <[email protected]>
        Date:   Sat Apr 27 08:57:55 2024 +0200

            [zscaler_zia] Fix mapping of source.ip and source.nat.ip (elastic#9727)

            * Fix mapping of source.ip and source.nat.ip

            * Update changelog

            * updated web datastream pipeline tests

            ---------

            Co-authored-by: Shourie Ganguly <[email protected]>

        commit 4750ea8
        Author: Mario Rodriguez Molins <[email protected]>
        Date:   Fri Apr 26 13:09:53 2024 +0200

            [nginx] Update nginx config to listen in ipv6 too (elastic#9720)

        commit 25b0988
        Author: Mario Rodriguez Molins <[email protected]>
        Date:   Fri Apr 26 10:45:03 2024 +0200

            [Buildkite] Update filter to use api source (elastic#9717)

        commit 45327cf
        Author: Mario Rodriguez Molins <[email protected]>
        Date:   Fri Apr 26 10:13:22 2024 +0200

            [Buildkite] Update filter condition to allow just from webhook source (elastic#9714)

        commit 024d864b49f1dd333529f96e06de6dec15aac703
        Author: milan-elastic <[email protected]>
        Date:   Fri Apr 26 13:00:47 2024 +0530

            add dashboard level filter for apache tomcat

        commit 1cb5fad
        Author: Dan Kortschak <[email protected]>
        Date:   Fri Apr 26 16:23:35 2024 +0930

            entityanalytics_ad: new package for Active Directory user collection (elastic#9485)

        commit 37c598f
        Author: CarsonHrusovsky <[email protected]>
        Date:   Thu Apr 25 18:13:26 2024 -0500

            [BBOT] New integration for Black Lantern Security scanner (elastic#9651)

        commit d13e474
        Author: Mario Rodriguez Molins <[email protected]>
        Date:   Thu Apr 25 11:55:39 2024 +0200

            [Buildkite] Skip install package command in serverless builds for some packages (elastic#9686)

        commit 0c2198b
        Author: Mario Rodriguez Molins <[email protected]>
        Date:   Thu Apr 25 11:41:42 2024 +0200

            [Buildkite] Add retry suffix for logs (elastic#9703)

        commit d932e79
        Author: Simon Kötting <[email protected]>
        Date:   Thu Apr 25 07:35:45 2024 +0200

            [Exchange Server] GA of Integration, Add Dashbord Panel Titles & System Tests (elastic#9560)

            * Add Dashboard Titles

            * Add Dashboard Titles

            * Change Version to GA

            * adjust PR in Changelog

            * Add System Tests to all datstreams

            * fix imap system test config

            * remove Folder structure out of system tests sample logs

            * Fix mapping

            * Add convert for inode field

            * specify numeric_keyword_fields in system tests

        commit dba2901
        Author: Dan Kortschak <[email protected]>
        Date:   Thu Apr 25 10:21:30 2024 +0930

            rapid7_insightvm: canonicalize host.name to lower case and map subdomain to host.hostname (elastic#9665)

        commit 4284262
        Author: Panos Koutsovasilis <[email protected]>
        Date:   Wed Apr 24 20:34:13 2024 +0300

            fix(fim): add auto option for backend and make it the default one (elastic#9702)

        commit c563bb3
        Author: Panos Koutsovasilis <[email protected]>
        Date:   Wed Apr 24 19:40:04 2024 +0300

            [juniper_netscreen]: include log.file.device_id and log.file.inode in base-fields (elastic#9658)

            * fix(juniper_netscreen): include log.file.device_id and log.file.inode in base-fields.yml

            * fix(juniper_netscreen): update README.md

        commit f187d0d
        Author: Panos Koutsovasilis <[email protected]>
        Date:   Wed Apr 24 19:11:28 2024 +0300

            [juniper_junos]: include log.file.device_id and log.file.inode in base-fields (elastic#9657)

            * fix(juniper_junos): include log.file.device_id and log.file.inode in base-fields.yml

            * fix(juniper_junos): update README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kpollich kpollich kpollich approved these changes

@jsoriano jsoriano jsoriano approved these changes

Assignees

@mrodm mrodm

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mrodm @elasticmachine @jsoriano @kpollich