Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Add CCR limitation #87348

Merged
merged 7 commits into from
Oct 17, 2022
Merged

[DOCS] Add CCR limitation #87348

Changes from 6 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
2877a8d
Add CCR limitation
Leaf-Lin Jun 3, 2022
f6196cf
Add restored index auto follow pattern restriction
Leaf-Lin Jun 14, 2022
aa7409e
Moving content to existing CCR page + several changes
Jul 7, 2022
4896110
Merge branch 'main' into Leaf-Lin-CCR-Limitation
elasticmachine Sep 8, 2022
aa1641f
Remove sections to consolidate limitation information
Oct 4, 2022
f8bcbeb
Delete separate file
Oct 4, 2022
dba1480
Remove restored indices from list of things that aren't replicated
Oct 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions docs/reference/ccr/index.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,33 @@ and use {ccr}:
* <<ccr-auto-follow>>
* <<ccr-upgrading>>

[discrete]
[[ccr-limitations]]
=== {ccr-cap} limitations
{ccr-cap} is designed to replicate user-generated indices only, and doesn't
currently replicate any of the following:

* <<system-indices,System indices>>
* {ml-docs}/machine-learning-intro.html[Machine learning jobs]
* <<index-templates,index templates>>
* <<index-lifecycle-management,{ilm-cap}>> and
<<snapshot-lifecycle-management-api,{slm}>> polices
* {ref}/mapping-roles.html[User permissions and role mappings]
* <<snapshots-register-repository,Snapshot repository settings>>
* <<modules-cluster,Cluster settings>>
* <<searchable-snapshots,Searchable snapshot>>
* <<restore-index-data-stream,Restored indices>>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this true, would auto-follow patterns not hit that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was referring to the problem described in this issue: #87055

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the conversation in #87055, it sounds like restored indices aren't replicated if the leader index is deleted and then restored, but the original replicated index still exists in the remote cluster. The restored index in the leader cluster gets a new UUID, so the auto-follow pattern on the remote cluster sees it as new index, but it's actually just a replicated index.

This sounds like an edge case that we could call attention to, but it seems like restored indices would be replicated if the underlying leader index is still intact. Is that true @henningandersen?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree @lockewritesdocs, it will follow just fine if the restore results in an unoccupied name on the follower. This is similar to any sort of naming conflict on the follower though a delete and restore on the leader side is possibly a common cause of this.

I'd opt to remove this, since it is not true that restored indices are not followed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Sounds good @henningandersen -- I've removed "Restored indices" from the list of things that aren't followed.


If you want to replicate any of this data, you must replicate it to a remote
cluster manually.

NOTE: Data for <<searchable-snapshots,searchable snapshot>> indices is stored in
the snapshot repository. {ccr-cap} won't replicate these indices completely, even
though they're either partially or fully-cached on the {es} nodes. To achieve
searchable snapshots in a remote cluster, configure snapshot repositories on
the remote cluster and use the same {ilm} policy from the local cluster to move
data into the cold or frozen tiers on the remote cluster.

include::getting-started.asciidoc[]
include::managing.asciidoc[]
include::auto-follow.asciidoc[]
Expand Down