[FEATURE]Add index administrative operations

[xluo-aws]

What / Why

What are you proposing? In a few sentences, describe the feature and its core capabilities.

*Answer: Users find running an OpenSearch cluster challenging in multiple ways. There is no interface for basic cluster administrative operations or to view cluster health metrics and status; instead customers have to use a wide range of stat and cat REST APIs to get insights into the cluster. If there is a problem, and ad hoc interventions need to be performed, customers either have to do this through REST APIs, which can be cumbersome to manage, or via YML configurations, which require cluster restarts. This lack of an out of box administration experience, makes it especially difficult for new users given the significant learning curve. With that said, we do have some administration user experiences (e.g. ISM, Alerting on HTTP inputs, Security, etc.). However, these are spread across a number of different plugin interfaces and so there is no unified administration experience for cluster administrators to go to and administer their deployments.

With this feature (Admin UI - Index Operations), we are launching a user interface for common indexing and data stream administrative operations in order to improve administrator ease of use. Functionality will include the following: Create, Read, Update, Delete (CRUD) and map indices, CRUD and map aliases, reindex, open/close indices, shrink indices, and split indices. Additionally, users will be able to monitor the actions taken within the UI through a dashboard, and audit records will be logged.

Long term, we plan to build upon this admin UI to establish a unified administration panel in OpenSearch Dashboards that will provide cluster administrators insights into their cluster health and metrics via pre-built dashboards, enable them to monitor and operate their clusters at one place, and an interface for some of the common cluster management operations (, force merge, shard reroute, snapshot, and more). It will also provide intelligent recommendations around sizing, security and more for healthy continuous operations. In the future, this interface can be extended to include more components like Data Prepper to provide a single place for full stack administration.

Which users have asked for this feature? ** (research, proposals, requests or anecdotes. Include links to GitHub Issues, Forums, Stack Overflow, Twitter, Etc.)

Answer:

• Create index templates
  • Create index templates directly in OpenSearch Dashboards OpenSearch-Dashboards#981
• Index deletion/operation:
  • Add more options to index management -> indices screen #154
  • [FEATURE] Add a button (in "Index Management" section?) to delete an index (if being admin) #142
• reindex:
  • Add reindexing to actions of the ILM index-management#190
  • (Remote) Reindex Action index-management#42
• Index setting update:
  • Modifying indices compression or static settings during lifecycle index-management#60
• Index alias:
  • Alias Action index-management#35

*What is the developer experience going to be? *Does this have a REST API? If so, please describe the API and any impact it may have to existing APIs. In a brief summary (not a spec), highlight what new REST APIs or changes to REST APIs are planned. as well as any other API, CLI or Configuration changes that are planned as part of this feature.

Answer: N/A this is a UI

_Are there any security considerations? _What is the security model of the new APIs? Features should be integrated into the OpenSearch security suite and so if they are not, we should highlight the reasons here.

Answer:
Auditing: Create/update/delete/Shrink/Split operation history can be tracked in audit log. Discover can be used to search the information.

Topic for discussion - Permissions:
Current Plan: Leverage existing permissions (https://opensearch.org/docs/latest/security-plugin/access-control/permissions/) or action groups that are enforced at API level. There is no UI-level permission control and we will not introduce new permissions or action groups. Whoever can access ISM is able to view the interface, but actions they take will fail if they do not have permissions to run the related APIs.

Proposed plan: Run pre-checks to establish a users ability to perform given actions and provide a clear UX to users who lack permission to perform certain actions (e.g. grey out actions they do not have permissions to take). In order to do this in a sustainable way (accounting for potential API permissions logic changes in the future), we will need to add the ability to send permission pre-checks from the APIs to the UI. This will increase the scope of the planned work, so we would like to discuss ideas to minimize this work with the broader group.

Are there any breaking changes to the API? If Yes, what is the path to minimizing impact? (example, add new API and deprecate the old one)

Answer: No

What is the user experience going to be?

Answer: Users will be able to manage their index operations within the index management interface. The guided experience helps users Create, Read, Update, Delete (CRUD) and map indices, CRUD and map aliases, reindex, open/close indices, shrink indices, and split indices. Additionally, users will be able to monitor the actions taken within the UI through a dashboard, and audit records will be logged.

Are there breaking changes to the User Experience?

Answer: We are replacing the “apply policy” button with an “actions” button which has the policy changes nested within it. This would only be a breaking change if a user has scripts written which rely on the UI elements. (Open question for discussion)

What will it take to execute? __ Are there any assumptions you may be making that could limit scope or add limitations? Are there performance, cost, or technical constraints that may impact the user experience? Does this feature depend on other feature work? What additional risks are there?

Answer:

• We will add CUD functions to the existing indices page within the Index Management plugin
• The majority of the effort will be to build the front end. The backend APIs already exist.
• We will introduce two new index statuses (reindex and recovery)
• Index mapping visual editor style will only support field name and type. If type has attribute, user need to use json editor.
• Limitation: An index template created through V1 API will not be displayed even if there is a match when creating an index.
• Operation history information will be tracked in an audit log. We can revisit later if there is a need to create a new page to track operation history.

_Any remaining open questions? _What are known enhancements to this feature? Any enhancements that may be out of scope but that we will want to track long term? List any other open questions that may need to be answered before proceeding with an implementation.

Answer:
No remaining open questions