Add segment replication + remote store GA information #5037

Naarcha-AWS · 2023-09-15T03:44:16Z

Resolves [DOC] Segment Replication + Remote Storage #4405
Resolves [DOC] Remote Store feature documentation changes for 2.10.0 release #4875
Resolves [DOC]Segment replication added consideration #4984

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS · 2023-09-15T03:48:53Z

@dreamer-89: This PR is ready for your review. Let me know if there is anything I missed.

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Signed-off-by: Naarcha-AWS <[email protected]>

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

…tion/index.md Co-authored-by: Suraj Singh <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Co-authored-by: Bukhtawar Khan <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Signed-off-by: Naarcha-AWS <[email protected]>

kolchfa-aws

Looks good but we need to reference a consistent name (either remote-backed storage or remote store).

_tuning-your-cluster/availability-and-recovery/remote-store/remote-segment-backpressure.md

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

Signed-off-by: Naarcha-AWS <[email protected]>

kolchfa-aws

LGTM

natebower

@Naarcha-AWS Please see my comments and changes and let me know if you have any questions. Thanks!

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

natebower · 2023-09-19T17:15:16Z

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

+As of OpenSearch 2.10, you can use two methods for segment replication:
+
+- With **remote-backed storage**, a persistent storage solution: The primary shard sends segment files to the remote-backed storage and the replica shards source the copy from the same store. For more information about using a remote-backed storage, see [Remote-backed storage]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/remote-store/index/#segment-replication-and-remote-backed-storage).
+- Without remote-backed storage: The primary shard sends segment files directly to the replica shards using node-to-node communication.


Can we call the second method what it is, as opposed to what it isn't? Node-to-node replication?

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

* Add segment replication + remote store GA information Signed-off-by: Naarcha-AWS <[email protected]> * Add introduction labels Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update _tuning-your-cluster/availability-and-recovery/segment-replication/index.md Co-authored-by: Suraj Singh <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Bukhtawar Khan <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update index.md Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Replace Remote Store with Remote Backed Storage Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> --------- Signed-off-by: Naarcha-AWS <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Co-authored-by: Suraj Singh <[email protected]> Co-authored-by: Bukhtawar Khan <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]>

This reverts commit 8d6bd01.

…ject#5037) * Add segment replication + remote store GA information Signed-off-by: Naarcha-AWS <[email protected]> * Add introduction labels Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update _tuning-your-cluster/availability-and-recovery/segment-replication/index.md Co-authored-by: Suraj Singh <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Bukhtawar Khan <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update index.md Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Replace Remote Store with Remote Backed Storage Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> --------- Signed-off-by: Naarcha-AWS <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Co-authored-by: Suraj Singh <[email protected]> Co-authored-by: Bukhtawar Khan <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]>

* Add segment replication + remote store GA information Signed-off-by: Naarcha-AWS <[email protected]> * Add introduction labels Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update _tuning-your-cluster/availability-and-recovery/segment-replication/index.md Co-authored-by: Suraj Singh <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Bukhtawar Khan <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <[email protected]> * Update index.md Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> * Replace Remote Store with Remote Backed Storage Signed-off-by: Naarcha-AWS <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> --------- Signed-off-by: Naarcha-AWS <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]> Co-authored-by: Suraj Singh <[email protected]> Co-authored-by: Bukhtawar Khan <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]>

Add segment replication + remote store GA information

96dc030

Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS requested review from cwillum, hdhalter, kolchfa-aws, vagimeli, ananzh, seanneumann, AMoo-Miki and natebower as code owners September 15, 2023 03:44

Add introduction labels

779369c

Signed-off-by: Naarcha-AWS <[email protected]>

anasalkouz reviewed Sep 15, 2023

View reviewed changes

hdhalter assigned Naarcha-AWS Sep 15, 2023

hdhalter added v2.10.0 release-notes PR: Include this PR in the automated release notes 3 - Tech review PR: Tech review in progress labels Sep 15, 2023

Naarcha-AWS commented Sep 15, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 15, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 15, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 15, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS added 2 commits September 15, 2023 12:27

Apply suggestions from code review

591ba8f

Signed-off-by: Naarcha-AWS <[email protected]>

Apply suggestions from code review

b34a484

Signed-off-by: Naarcha-AWS <[email protected]>

dreamer-89 suggested changes Sep 15, 2023

View reviewed changes

Update _tuning-your-cluster/availability-and-recovery/segment-replica…

9b668fb

…tion/index.md Co-authored-by: Suraj Singh <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

Bukhtawar reviewed Sep 16, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Bukhtawar reviewed Sep 16, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 18, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 18, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Apply suggestions from code review

c5291f7

Co-authored-by: Bukhtawar Khan <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS requested a review from Bukhtawar September 18, 2023 17:16

Naarcha-AWS requested review from dreamer-89 and anasalkouz September 18, 2023 17:16

Naarcha-AWS commented Sep 18, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS commented Sep 18, 2023

View reviewed changes

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved

Naarcha-AWS added 2 commits September 18, 2023 12:28

Apply suggestions from code review

92133af

Signed-off-by: Naarcha-AWS <[email protected]>

Update index.md

a2e3323

Signed-off-by: Naarcha-AWS <[email protected]>

kolchfa-aws reviewed Sep 19, 2023

View reviewed changes

Apply suggestions from code review

0fc62f0

Co-authored-by: kolchfa-aws <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS requested a review from kolchfa-aws September 19, 2023 15:35

Naarcha-AWS added 4 - Doc review PR: Doc review in progress and removed 3 - Tech review PR: Tech review in progress labels Sep 19, 2023

Replace Remote Store with Remote Backed Storage

f4c8df8

Signed-off-by: Naarcha-AWS <[email protected]>

kolchfa-aws approved these changes Sep 19, 2023

View reviewed changes

Naarcha-AWS added 5 - Editorial review PR: Editorial review in progress and removed 4 - Doc review PR: Doc review in progress labels Sep 19, 2023

natebower reviewed Sep 19, 2023

View reviewed changes

Apply suggestions from code review

1b215d2

Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS merged commit 64e120a into main Sep 19, 2023
4 checks passed

vagimeli added a commit that referenced this pull request Sep 19, 2023

Revert "Add segment replication + remote store GA information (#5037)"

c71fb7a

This reverts commit 8d6bd01.

dreamer-89 mentioned this pull request Sep 25, 2023

[DOC] Update segment replication consideration section #5102

Closed

4 tasks

Naarcha-AWS deleted the segment-replication-ga branch March 28, 2024 23:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add segment replication + remote store GA information #5037

Add segment replication + remote store GA information #5037

Naarcha-AWS commented Sep 15, 2023

Naarcha-AWS commented Sep 15, 2023

kolchfa-aws left a comment

kolchfa-aws left a comment

natebower left a comment

natebower Sep 19, 2023

Add segment replication + remote store GA information #5037

Add segment replication + remote store GA information #5037

Conversation

Naarcha-AWS commented Sep 15, 2023

Checklist

Naarcha-AWS commented Sep 15, 2023

kolchfa-aws left a comment

Choose a reason for hiding this comment

kolchfa-aws left a comment

Choose a reason for hiding this comment

natebower left a comment

Choose a reason for hiding this comment

natebower Sep 19, 2023

Choose a reason for hiding this comment