Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Make ML indices hidden when the node becomes master #77416

Merged
merged 18 commits into from
Oct 11, 2021

Conversation

przemekwitek
Copy link
Contributor

@przemekwitek przemekwitek commented Sep 8, 2021

This PR makes the ML indices hidden when the node becomes master.
The affected ML indices are: .ml-anomalies-*, .ml-state*, .ml-stats-*, .ml-notifications*, .ml-annotations*.
This change is needed because these indices could have been created before the notion of the hidden index was introduced.

Relates #53674

@cla-checker-service
Copy link

cla-checker-service bot commented Sep 8, 2021

💚 CLA has been signed

@przemekwitek przemekwitek force-pushed the make_indices_hidden branch 5 times, most recently from e2d5bd7 to 7954500 Compare September 15, 2021 07:46
@przemekwitek przemekwitek changed the title [DRAFT] New test [ML] Make ML indices hidden when the node becomes master Sep 15, 2021
@przemekwitek przemekwitek added :ml Machine learning >enhancement v7.16.0 and removed WIP labels Sep 15, 2021
@przemekwitek przemekwitek marked this pull request as ready for review September 15, 2021 11:17
@elasticmachine elasticmachine added the Team:ML Meta label for the ML team label Sep 15, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

@przemekwitek przemekwitek force-pushed the make_indices_hidden branch 3 times, most recently from 3c3e2b5 to 5564011 Compare September 21, 2021 08:19
@przemekwitek przemekwitek force-pushed the make_indices_hidden branch 3 times, most recently from dcf5457 to fd2a606 Compare October 5, 2021 06:21
Copy link
Contributor Author

@przemekwitek przemekwitek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aliases are now made hidden too.

PTAL

Copy link
Contributor

@droberts195 droberts195 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like there might be a couple of things left over from debugging, and I also left a couple of comments about logging.

Copy link
Contributor

@droberts195 droberts195 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@przemekwitek
Copy link
Contributor Author

run elasticsearch-ci/part-1

@przemekwitek przemekwitek merged commit 0a6b6d9 into elastic:master Oct 11, 2021
@przemekwitek przemekwitek deleted the make_indices_hidden branch October 11, 2021 09:55
weizijun added a commit to weizijun/elasticsearch that referenced this pull request Oct 11, 2021
* upstream/master: (250 commits)
  [Transform] HLRC cleanups (elastic#78909)
  [ML] Make ML indices hidden when the node becomes master (elastic#77416)
  Introduce a Few Settings Singleton Instances (elastic#78897)
  Simplify TestCluster extraJar configuration (elastic#78837)
  Add @OverRide annotations to methods in EnrichPlugin class (elastic#76873)
  Add v7 restCompat for invalidating API key with the id field (elastic#78664)
  EQL: Refine repeatable queries (elastic#78895)
  Fix DataTierTests package and add a validation test (elastic#78880)
  Fix split package org.elasticsearch.common.xcontent (elastic#78831)
  Store DataTier Preference directly on IndexMetadata (elastic#78668)
  [DOCS] Fixes typo in calendar API example (elastic#78867)
  Improve Node Shutdown Observability (elastic#78727)
  Convert encrypted snapshot license object to LicensedFeature (elastic#78731)
  Revert "Make nodePaths() singular (elastic#72514)" (elastic#78801)
  Fix incorrect generic type in PolicyStepsRegistry (elastic#78628)
  [DOCS] Fixes ML get calendars API (elastic#78808)
  Implement GET API for System Feature Upgrades (elastic#78642)
  [TEST] More MetadataStateFormat tests (elastic#78577)
  Add support for rest compatibility headers to the HLRC (elastic#78490)
  Un-ignoring tests after backporting fix (elastic#78830)
  ...

# Conflicts:
#	server/src/main/java/org/elasticsearch/ingest/IngestService.java
#	server/src/test/java/org/elasticsearch/ingest/IngestServiceTests.java
droberts195 added a commit to droberts195/elasticsearch that referenced this pull request Nov 1, 2021
This is a followup to elastic#77416.

If aliases on ML hidden indices are made hidden then
all attributes of the alias need to be preserved.

The particularly critical case for ML is where a job's
results alias has a filter that restricts documents
returned when searching the shared results index to
those that relate to the job.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :ml Machine learning Team:ML Meta label for the ML team v8.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants