-
Notifications
You must be signed in to change notification settings - Fork 24.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Docs for snapshots as simple archives (#87211)
Backport of #86261
- Loading branch information
Showing
3 changed files
with
109 additions
and
15 deletions.
There are no files selected for viewing
19 changes: 10 additions & 9 deletions
19
docs/reference/snapshot-restore/cluster-index-compat.asciidoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,13 @@ | ||
|
|
||
| [cols="^,^,^,^,^"] | ||
| [cols="^,^,^,^,^,^"] | ||
| |==== | ||
| | 4+^h| Cluster version | ||
| h| Index creation version | 6.8 | 7.0–7.1 | 7.2–{prev-major-last} | 8.0–{minor-version} | ||
| | 5.0–5.6 | {yes-icon} | {no-icon} | {no-icon} | {no-icon} | ||
| | 6.0–6.7 | {yes-icon} | {yes-icon} | {yes-icon} | {no-icon} | ||
| | 6.8 | {yes-icon} | {no-icon} | {yes-icon} | {no-icon} | ||
| | 7.0–7.1 | {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | ||
| | 7.2–{prev-major-last} | {no-icon} | {no-icon} | {yes-icon} | {yes-icon} | ||
| | 8.0–{minor-version} | {no-icon} | {no-icon} | {no-icon} | {yes-icon} | ||
| | 5+^h| Cluster version | ||
| h| Index creation version | 6.8 | 7.0–7.1 | 7.2–{prev-major-last} | 8.0–8.2 | 8.3-{minor-version} | ||
| | 5.0–5.6 | {yes-icon} | {no-icon} | {no-icon} | {no-icon} | {yes-icon}footnote:archive[Using <<archive-indices,archive>> functionality] | ||
| | 6.0–6.7 | {yes-icon} | {yes-icon} | {yes-icon} | {no-icon} | {yes-icon}footnote:archive[] | ||
| | 6.8 | {yes-icon} | {no-icon} | {yes-icon} | {no-icon} | {yes-icon}footnote:archive[] | ||
| | 7.0–7.1 | {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | {yes-icon} | ||
| | 7.2–{prev-major-last} | {no-icon} | {no-icon} | {yes-icon} | {yes-icon} | {yes-icon} | ||
| | 8.0–8.2 | {no-icon} | {no-icon} | {no-icon} | {yes-icon} | {yes-icon} | ||
| | 8.3–{minor-version} | {no-icon} | {no-icon} | {no-icon} | {yes-icon} | {yes-icon} | ||
| |==== |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| [[archive-indices]] | ||
| == Reading indices from older {es} versions | ||
|
|
||
| {es} has full query and write support for indices created in the previous major | ||
| version. If you have indices created in {es} versions 5 or 6, you can now use | ||
| the archive functionality to import them into newer {es} versions as well. | ||
|
|
||
| The archive functionality provides slower read-only access to older {es} data, | ||
| for compliance or regulatory reasons, the occasional lookback or investigation, | ||
| or to rehydrate parts of it. Access to the data is expected to be infrequent, | ||
| and can therefore happen with limited performance and query capabilities. | ||
|
|
||
| For this, {es} has the ability to access older snapshot repositories | ||
| (going back to version 5). The legacy indices in the <<snapshot-restore,snapshot repository>> | ||
| can either be <<restore-snapshot-api,restored>>, or can be directly accessed | ||
| via <<searchable-snapshots,searchable snapshots>> so that the archived data | ||
| won't even need to fully reside on local disks for access. | ||
|
|
||
| [discrete] | ||
| [[archive-indices-supported-field-types]] | ||
| === Supported field types | ||
|
|
||
| Old mappings are imported as much "as-is" as possible into {es} 8, but only | ||
| provide regular query capabilities on a select subset of fields: | ||
|
|
||
| - <<number,Numeric types>> | ||
| - <<boolean,`boolean` type>> | ||
| - <<ip,`ip` type>> | ||
| - <<geo-point,`geo_point` type>> | ||
| - <<date,`date` types>>: the date `format` setting on date fields is supported | ||
| as long as it behaves similarly across these versions. In case it is not, | ||
| for example {ref-7x}/migrate-to-java-time.html[when using custom date formats], | ||
| this field can be updated on legacy indices so that it can be changed by a | ||
| user if need be. | ||
| - <<keyword-field-type,`keyword` type>>: the `normalizer` setting on keyword | ||
| fields is supported as long as it behaves similarly across these versions. | ||
| In case it is not, this field can be updated on legacy indices if need be. | ||
| - <<text-field-type,`text` type>>: scoring capabilities are limited, and all | ||
| queries return constant scores that are equal to 1.0. The `analyzer` | ||
| settings on text fields are supported as long as they behave similarly | ||
| across these versions. In case they do not, they can be updated on legacy | ||
| indices if need be. | ||
| - <<multi-fields,Multi-fields>> | ||
| - <<field-alias,Field aliases>> | ||
| - <<object,`object`>> fields | ||
| - some basic metadata fields, e.g. `_type` for querying {es} 5 indices | ||
| - <<runtime-mapping-fields,runtime fields>> | ||
| - <<mapping-source-field,`_source` field>> | ||
|
|
||
| {es} 5 indices with mappings that have {ref-7x}/removal-of-types.html[multiple mapping types] | ||
| are collapsed together on a best-effort basis before they are imported. | ||
|
|
||
| In case the auto-import of mappings does not work, or the new {es} version | ||
| can't make sense of the mapping, it falls back to importing the index without | ||
| the mapping, but stores the original mapping in the <<mapping-meta-field,_meta>> | ||
| section of the imported index. The legacy mapping can then be introspected | ||
| using the <<indices-get-mapping,GET mapping>> API and an updated mapping can | ||
| be manually put in place using the <<indices-put-mapping,update mapping>> API, | ||
| copying and adapting relevant sections of the legacy mapping to work with the | ||
| current {es} version. While auto-import is expected to work in most cases, | ||
| failures of doing so should be https://github.com/elastic/elasticsearch/issues/new/choose[raised] | ||
| with the Elastic team for future improvements. | ||
|
|
||
| [discrete] | ||
| === Supported APIs | ||
|
|
||
| Archive indices are read-only, and provide data access via the | ||
| <<search-search,search>> and <<search-field-caps,field capabilities>> APIs. | ||
| They do not support the <<docs-get,Get API>> nor any write APIs. | ||
|
|
||
| Archive indices allow running queries as well as aggregations in so far as | ||
| they are <<archive-indices-supported-field-types,supported by the given field type>>. | ||
|
|
||
| Due to `_source` access the data can also be <<docs-reindex,reindexed>> | ||
| to a new index that has full compatibility with the current {es} version. | ||
|
|
||
| [discrete] | ||
| === How to upgrade older {es} 5 or 6 clusters? | ||
|
|
||
| Take a snapshot of the indices in the old cluster, delete indices that are not | ||
| directly supported by ES 8 (i.e. indices older than 7.0), upgrade the cluster | ||
| without the old indices, and then <<restore-snapshot-api,restore>> the legacy | ||
| indices from the snapshot or <<searchable-snapshots-api-mount-snapshot,mount>> | ||
| them via searchable snapshots. | ||
|
|
||
| In the future, we plan on streamlining the upgrade process going forward, | ||
| making it easier to take legacy indices along when going to future major | ||
| {es} versions. |