[Upgrade Assistant] Removes unnecessary mappings for reindexing

[yuliacech]

Summary

Partially addresses #64547

This PR removes unnecessary mappings from the saved objects type upgrade-assistant-reindex-operation that holds information about reindex processes.
Currently, the reindexing is disabled via a config for Upgrade Assistant plugin xpack.upgrade_assistant.featureSet.reindexCorrectiveActions. ES also doesn't log any deprecations for indices created since 7.0 and indices created before 7.0 are incompatible with versions 8.0+. That means that when reindexing will be enabled in UA, it needs to be configured for the specific upgrade scenario, for example for 9.0.

How to test

Even though reindexing is currently disabled, I was able to do some testing for the changes. Here is how to test reindexing locally:

1. Add xpack.upgrade_assistant.featureSet.reindexCorrectiveActions: true to your kibana.dev.yml file
2. Start ES and Kibana with yarn es snapshot --license=trial and yarn start
3. Create a test user with the role system_indices_superuser (in Stack Management -> Security -> Users). Log in with that user for further steps.
4. Create an index via Dev Tools

POST /old_index/_doc
{
  "test": "test"
}

5. replace deprecations on line 22 in the file /server/lib/es_deprecations_status.ts with

  const deprecations = {
    cluster_settings: [],
    node_settings: [],
    ml_settings: [],
    index_settings: {
      old_index: [
        {
          level: 'critical',
          message: 'Index created before 7.0',
          url: 'https: //www.elastic.co/guide/en/elasticsearch/reference/master/breaking-changes-8.0.html',
          details: 'This index was created using version: 6.8.13',
          resolve_during_rolling_upgrade: false,
        },
      ],
    },
  };

6. open UA and test the reindexing of old_index

Follow up work

A follow up PR if needed will be opened for saved objects types upgrade-assistant-ml-upgrade-operation and upgrade-assistant-telemetry

[elasticmachine]

Pinging @elastic/platform-deployment-management (Team:Deployment Management)

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: f44a45f

Metrics [docs]

Saved Objects .kibana field count

Every field in each saved object type adds overhead to Elasticsearch. Kibana needs to keep the total field count below Elasticsearch's default limit of 1000 fields. Only specify field mappings for the fields you wish to search on or query. See https://www.elastic.co/guide/en/kibana/master/saved-objects-service.html#_mappings

id	before	after	diff
upgrade-assistant-reindex-operation	18	3	-15

History

• 💔 Build #107536 failed 5ff22dd
• 💔 Build #107520 failed 51ef046
• 💔 Build #107503 failed 2655e98
• 💔 Build #107260 failed c4fb200

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[ElenaStoeva]

Thanks for working on this @yuliacech! Checked out the PR locally and tested reindexing - works fine for me. 👍

Regarding the code changes, I see that some of the deleted fields are used in multiple test files and other files like x-pack/plugins/upgrade_assistant/server/lib/reindexing/reindex_actions.ts for example:

    async createReindexOp(indexName: string, opts?: ReindexOptions) {
      return client.create<ReindexOperation>(REINDEX_OP_TYPE, {
        indexName,
        newIndexName: generateNewIndexName(indexName),
        status: ReindexStatus.inProgress,
        lastCompletedStep: ReindexStep.created,
        locked: null,
        reindexTaskId: null,
        reindexTaskPercComplete: null,
        errorMessage: null,
        runningReindexCount: null,
        reindexOptions: opts,
      });
    },

Should these files be updated as well?

[yuliacech]

Thanks a lot for your review, @ElenaStoeva!
The fields this PR removes are only in the mappings, that means the code won't be able to query or search saved objects by this fields. For example, as it's done in the linked file on line 174:

findReindexOperations(indexName: string) {
      return client.find<ReindexOperation>({
        type: REINDEX_OP_TYPE,
        search: `"${indexName}"`,
        searchFields: ['indexName'],
      });
    },

For this code to work, we need to keep the field indexName in the mappings for reindex operation saved objects. The same is true for the field status (line 185).
The rest of the properties will still be stored in saved objects but only as "not indexed" content so to say. The Core team recommends to consider if the fields are being searched, sorted or aggregated over. Be sure to consider queries from browser side, server side, public HTTP APIs and telemetry collection.

[ElenaStoeva]

Thanks a lot for your review, @ElenaStoeva! The fields this PR removes are only in the mappings, that means the code won't be able to query or search saved objects by this fields. For example, as it's done in the linked file on line 174:

findReindexOperations(indexName: string) {
      return client.find<ReindexOperation>({
        type: REINDEX_OP_TYPE,
        search: `"${indexName}"`,
        searchFields: ['indexName'],
      });
    },

For this code to work, we need to keep the field indexName in the mappings for reindex operation saved objects. The same is true for the field status (line 185). The rest of the properties will still be stored in saved objects but only as "not indexed" content so to say. The Core team recommends to consider if the fields are being searched, sorted or aggregated over. Be sure to consider queries from browser side, server side, public HTTP APIs and telemetry collection.

Thanks for the clarification @yuliacech! Reviewed the code changes again and given what you explained, everything looks good to me. 👍