Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discovery GCE Pagination Lack #83783

Closed
cdmbr opened this issue Feb 10, 2022 · 1 comment · Fixed by #83785
Closed

Discovery GCE Pagination Lack #83783

cdmbr opened this issue Feb 10, 2022 · 1 comment · Fixed by #83785
Labels
>bug :Distributed Coordination/Discovery-Plugins Anything related to our integration plugins with EC2, GCP and Azure Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.

Comments

@cdmbr
Copy link
Contributor

cdmbr commented Feb 10, 2022

Elasticsearch Version

All

Installed Plugins

discovery-gce

Java Version

bundled

OS Version

any

Problem Description

Currently, discovery-gce plugin tries to obtain all instances for a given project/zone in one shot.
Recently I've faced in my company (that had ~ 700 instances running at the time) a weird behavior of this plugin. The google api that this plugin uses paginates results in chunks of 500 items. The current implementation is ignoring if there's a next page available.

Steps to Reproduce

  • run the node discovery process in zones that have more than 500 instances

Logs (if relevant)

No response

@cdmbr cdmbr added >bug needs:triage Requires assignment of a team area label labels Feb 10, 2022
@DaveCTurner DaveCTurner added the :Distributed Coordination/Discovery-Plugins Anything related to our integration plugins with EC2, GCP and Azure label Feb 10, 2022
@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Feb 10, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@DaveCTurner DaveCTurner added Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. and removed Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. needs:triage Requires assignment of a team area label labels Feb 10, 2022
arteam pushed a commit that referenced this issue Feb 15, 2022
#83785)

Discovery gce plugin has some issues while running in large pools of vms instances.

This pr attempts to solve it.

Closes #83783
arteam pushed a commit to arteam/elasticsearch that referenced this issue Feb 15, 2022
elastic#83785)

Discovery gce plugin has some issues while running in large pools of vms instances.

This pr attempts to solve it.

Closes elastic#83783
arteam pushed a commit to arteam/elasticsearch that referenced this issue Feb 15, 2022
elastic#83785)

Discovery gce plugin has some issues while running in large pools of vms instances.

This pr attempts to solve it.

Closes elastic#83783
arteam added a commit that referenced this issue Feb 15, 2022
#83785) (#83978)

Discovery gce plugin has some issues while running in large pools of vms instances.

This pr attempts to solve it.

Closes #83783

Co-authored-by: Claudio Marins <[email protected]>
arteam added a commit that referenced this issue Feb 15, 2022
#83785) (#83979)

Discovery gce plugin has some issues while running in large pools of vms instances.

This pr attempts to solve it.

Closes #83783

Co-authored-by: Claudio Marins <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Distributed Coordination/Discovery-Plugins Anything related to our integration plugins with EC2, GCP and Azure Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants