Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Anomaly Detection Create components integration with HashRing #200

Open
joshpalis opened this issue Oct 19, 2022 · 1 comment
Open

Anomaly Detection Create components integration with HashRing #200

joshpalis opened this issue Oct 19, 2022 · 1 comment
Assignees
Labels
discuss enhancement New feature or request

Comments

@joshpalis
Copy link
Member

joshpalis commented Oct 19, 2022

Is your feature request related to a problem?

Create components integration for Anomaly Detection requires the removal of ClusterService from component constructors and to replace them with transport API requests to OpenSearch to add setting update consumers to OpenSearch's ClusterService. However, the ADTaskManager component requires this HashRing object.

Internally, this HashRing utilizes the cluster service to determine the AD version of the local node and either build or retrieve nodes with the same version.

This is used by the ADTaskManager to identify and send requests to the coordinating Anomaly Detector node, which in turn delegates AD tasks to worker nodes to execute.

Open to suggestions on how to move forward. CC: @saratvemulapalli @owaiskazi19 @dbwiddis

@joshpalis joshpalis added enhancement New feature or request untriaged labels Oct 19, 2022
@joshpalis joshpalis self-assigned this Oct 19, 2022
@dbwiddis
Copy link
Member

The underlying issue is that in the plugin model, a copy of the plugin is running on every node in the OpenSearch cluster, so the cluster state is relevant for finding those other same-version plugin copies.

In the extensions model, we are executing AD tasks locally on the extension and without a corresponding "plugin" running on other nodes, there's no reason to send anything to the ExtensionsOrchestrator.

The ideal long term solution is to have a standard "execute this task on the OpenSearch cluster" with the ability to control which nodes in the cluster execute it based on some filter. In the near term, I think we need to ignore this "check AD version" bit and just send everything to the whole cluster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discuss enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants