From 724d91e5af1da279ec37acd1c7bf2f8b0d86ac72 Mon Sep 17 00:00:00 2001 From: Gerard Soldevila Date: Tue, 23 May 2023 15:53:52 +0200 Subject: [PATCH] Update saved objects migrations documentation for 8.8.0 (#158241) Documentation must reflect changes brought by the [dot kibana split](https://github.com/elastic/kibana/pull/154888). Our public facing documentation just mentions two indices e.g. https://www.elastic.co/guide/en/kibana/8.8/saved-object-migrations.html https://www.elastic.co/guide/en/kibana/8.8/resolve-migrations-failures.html#upgrade-migrations-old-indices this one in particular mentions that "all other indices are safe to delete". (cherry picked from commit 70eb9d3ecbb11768d091fca385bad4f20960deef) --- .../resolving-migration-failures.asciidoc | 12 +--- .../upgrade/saved-objects-migration.asciidoc | 62 ++++++++++++++----- 2 files changed, 49 insertions(+), 25 deletions(-) diff --git a/docs/setup/upgrade/resolving-migration-failures.asciidoc b/docs/setup/upgrade/resolving-migration-failures.asciidoc index 63264cb4f0963..d81a577a19a88 100644 --- a/docs/setup/upgrade/resolving-migration-failures.asciidoc +++ b/docs/setup/upgrade/resolving-migration-failures.asciidoc @@ -8,8 +8,8 @@ with the new version. ==== Saved object migration failures If {kib} unexpectedly terminates while migrating a saved object index, {kib} automatically attempts to -perform the migration again when the process restarts. Do not delete any saved objects indices to -fix a failed migration. Unlike previous versions, {kib} 7.12.0 and later does not require deleting +perform the migration again when the process restarts. Do not delete any saved objects indices to +fix a failed migration. Unlike previous versions, {kib} 7.12.0 and later does not require deleting indices to release a failed migration lock. If upgrade migrations fail repeatedly, refer to @@ -19,14 +19,6 @@ When you address the root cause for the migration failure, If you're unable to resolve a failed migration, contact Support. -[float] -[[upgrade-migrations-old-indices]] -==== Old `.kibana_N` indices - -After the migrations complete, multiple {kib} indices are created in {es}: (`.kibana_1`, `.kibana_2`, `.kibana_7.12.0` etc). -{kib} only uses the index that the `.kibana` and `.kibana_task_manager` aliases point to. -The other {kib} indices can be safely deleted, but are left around as a matter of historical record, and to facilitate rolling {kib} back to a previous version. - [float] ==== Known issues with {fleet} beta If you see a`timeout_exception` or `receive_timeout_transport_exception` error, diff --git a/docs/setup/upgrade/saved-objects-migration.asciidoc b/docs/setup/upgrade/saved-objects-migration.asciidoc index 3dac6a02c96ef..1f4ed9a7fff09 100644 --- a/docs/setup/upgrade/saved-objects-migration.asciidoc +++ b/docs/setup/upgrade/saved-objects-migration.asciidoc @@ -8,34 +8,66 @@ To access the assistant, go to *Stack Management > Upgrade Assistant*. WARNING: {kib} 7.12.0 and later uses a new migration process and index naming scheme. Before you upgrade, read the documentation for your version of {kib}. -WARNING: The following instructions assumes {kib} is using the default index names. If the `kibana.index` or `xpack.tasks.index` configuration settings are different from the default, adapt the instructions accordingly. +WARNING: The `kibana.index` and `xpack.tasks.index` configuration settings are obsolete and no longer taken into account in 8.x. If you are using custom index names, please perform the necessary adaptations before attempting to upgrade to 8.x. [float] [[upgrade-migrations-process]] ==== How saved objects migrations work -Saved objects are stored in two indices: - -* `.kibana_{kibana_version}_001`, e.g. for {kib} 7.12.0 `.kibana_7.12.0_001`. -* `.kibana_task_manager_{kibana_version}_001`, e.g. for {kib} 7.12.0 `.kibana_task_manager_7.12.0_001`. - -The index aliases `.kibana` and `.kibana_task_manager` always point to -the most up-to-date saved object indices. - When you start a new {kib} installation, an upgrade migration is performed before starting plugins or serving HTTP traffic. Before you upgrade, shut down old nodes to prevent losing acknowledged writes. To reduce the likelihood of old nodes losing acknowledged writes, {kib} 7.12.0 and later adds a write block to the outdated index. -The following tables lists the saved objects indices used by previous {kib} versions. +Saved objects are stored in multiple indices. Whilst all of them start with the `.kibana*` prefix, other `.kibana*` indices exist, which are not used to store saved objects. The following tables lists the saved objects indices used by each {kib} version. .Saved object indices and aliases per {kib} version [options="header"] |======================= -|Upgrading from version | Outdated index (alias) -| 6.5.0 through 7.3.x | `.kibana_N` (`.kibana` alias) +|Upgrading from version | Index | Aliases +| 6.5.0 through 7.3.x +| `.kibana_N` +| `.kibana` | 7.4.0 through 7.11.x -| `.kibana_N` (`.kibana` alias) - -`.kibana_task_manager_N` (`.kibana_task_manager` alias) +| `.kibana_N` + +`.kibana_task_manager_N` +| `.kibana` + +`.kibana_task_manager` +| 7.11.x through 8.7.x +| `.kibana_{kibana_version}_001` + +`.kibana_task_manager_{kibana_version}_001` +| `.kibana`, `.kibana_{kibana_version}` + +`.kibana_task_manager`, `.kibana_task_manager_{kibana_version}` +| 8.8.0+ +| `.kibana_{kibana_version}_001` + +`.kibana_alerting_cases_{kibana_version}_001` +`.kibana_analytics_{kibana_version}_001` +`.kibana_ingest_{kibana_version}_001` +`.kibana_task_manager_{kibana_version}_001` +`.kibana_security_solution_{kibana_version}_001` +| `.kibana`, `.kibana_{kibana_version}` + +`.kibana_alerting_cases`, `.kibana_alerting_cases_{kibana_version}` +`.kibana_analytics`, `.kibana_analytics_{kibana_version}` +`.kibana_ingest`, `.kibana_ingest_{kibana_version}` +`.kibana_task_manager`, `.kibana_task_manager_{kibana_version}` +`.kibana_security_solution`, `.kibana_security_solution_{kibana_version}` |======================= + +Starting on 7.11.0, each of the saved objects indices has a couple of aliases, e.g. the `.kibana_8.8.0_001` index has a _default_ alias `.kibana` and a _version_ alias `.kibana_8.8.0`. The _default_ aliases (e.g. `.kibana` and `.kibana_task_manager`) always point to +the most up-to-date saved object indices. Then, _version_ aliases are aligned with the deployed {kib} version. + + +Starting on 8.6.0, index names aren't necessarily aligned with the deployed {kib} version. When updates on a certain index are compatible, {kib} will keep the existing index instead of creating a new one. This allows for a more efficient upgrade process. The following example illustrates a completely valid state for a 8.8.0 deployment: + +* `.kibana_8.8.0_001` index, with `.kibana` and `.kibana_8.8.0` aliases. +* `.kibana_task_manager_8.7.0_001` index, with `.kibana_task_manager` and `.kibana_task_manager_8.8.0` aliases. + +Starting on 8.8.0, {kib} splits the main saved object index into multiple ones, as depicted on the table above. When upgrading from a previous version, the {kib} migration process will reindex some saved objects from the `.kibana` index into the new indices, depending on their types. Note that the `.kibana` index still exists, and it continues to store multiple saved object types. + +[float] +[[upgrade-migrations-old-indices]] +==== Old {kib} indices + +As a deployment is gradually upgraded, multiple {kib} indices are created in {es}: (`.kibana_1`, `.kibana_2`, `.kibana_7.12.0_001`, `.kibana_7.13.0_001`, `.kibana_8.0.0_001` etc). +{kib} only uses those indices that the _default_ and _version_ aliases point to. +The other, older {kib} saved object indices can be safely deleted, but are left around as a matter of historical record, and to facilitate rolling {kib} back to a previous version.