[Remote Store] Add file details to recoveryState while downloading segments from remote store #6825
+99
−25
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Gradle Check (Jenkins) Run Completed with:
|
|
Sample response: |
sachinpkale
requested review from
reta,
anasalkouz,
andrross,
Bukhtawar,
CEHENKLE,
dblock,
gbbafna,
setiah,
kartg,
kotwanikunal,
mch2,
nknize,
owaiskazi19,
Rishikesh1159,
ryanbogan,
saratvemulapalli,
shwetathareja,
dreamer-89,
tlfeng,
VachaShah and
xuezhou25
as code owners
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more @@ Coverage Diff @@
## main #6825 +/- ##
============================================
+ Coverage 70.60% 70.68% +0.08%
- Complexity 59151 59228 +77
============================================
Files 4810 4810
Lines 283502 283515 +13
Branches 40884 40888 +4
============================================
+ Hits 200153 200414 +261
+ Misses 66869 66635 -234
+ Partials 16480 16466 -14
... and 505 files with indirect coverage changes Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
ashking94
reviewed
Mar 28, 2023
| @@ -4377,6 +4377,9 @@ public void close() throws IOException { | |||
| * @throws IOException if exception occurs while reading segments from remote store | |||
| */ | |||
| public void syncSegmentsFromRemoteSegmentStore(boolean overrideLocal) throws IOException { | |||
| if (recoveryState.getStage() != RecoveryState.Stage.INDEX) { | |||
There was a problem hiding this comment.
nit - should we add an assert here and control this from the caller itself; what I mean is that calling this method when recovery state stage is not Index is probably not the correct thing to do in the first place. By the looks, for the caller it could lead to unexpected behaviour while this call is supposed to sync segments from remote store.
There was a problem hiding this comment.
Agree, I am not very much convinced on adding this block here (also the part of quietly returning). In SegRep integration with remote store, this can create issues (cc: @ankitkala ). Let me think more on this.
ashking94
reviewed
Mar 28, 2023
| @@ -268,7 +268,9 @@ public void copyFrom(Directory from, String src, String dest, IOContext context) | |||
| in.copyFrom(new FilterDirectory(from) { | |||
| @Override | |||
| public IndexInput openInput(String name, IOContext context) throws IOException { | |||
| index.addFileDetail(dest, l, false); | |||
| if (index.getFileDetails(dest) == null) { | |||
There was a problem hiding this comment.
Just a question - this check has been added to dedup file download? or from stats point of view?
There was a problem hiding this comment.
Why do we need to add this ? Is it for retries for the same file ?
There was a problem hiding this comment.
This check is from stats point of view. If I report the same file twice, assertion in addFileDetails method fails.
Before downloading files, we are reporting all the files to be downloaded (new code block is added in the IndexShard class) before the actual transfer starts.
There was a problem hiding this comment.
Should we then remove the assert ? Looks to be harmless .
There was a problem hiding this comment.
Those asserts make sure that we don't add the same file twice to the recovery tracker. IMO, that is the correct behavior. Currently, in our restore flow, we call IndexShard.syncSegmentsFromRemoteSegmentStore twice. Even though the second time, we skip downloading the files, without the above check, it will get added to tracker again.
ashking94
reviewed
Mar 28, 2023
added 6 commits
March 30, 2023 12:58
…ote store Signed-off-by: Sachin Kale <[email protected]>
Signed-off-by: Sachin Kale <[email protected]>
Signed-off-by: Sachin Kale <[email protected]>
Signed-off-by: Sachin Kale <[email protected]>
Signed-off-by: Sachin Kale <[email protected]>
Signed-off-by: Sachin Kale <[email protected]>
sachinpkale
force-pushed
the
recovery-file-details
branch
from
4306497 to
f15b02d
Compare
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Sachin Kale <[email protected]>
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Sachin Kale <[email protected]>
Gradle Check (Jenkins) Run Completed with:
|
gbbafna
approved these changes
Apr 4, 2023
gbbafna
approved these changes
Apr 4, 2023
opensearch-trigger-bot bot
pushed a commit
that referenced
this pull request
Apr 4, 2023
…gments from remote store (#6825) * Use existing StatsDirectoryWrapper to record recovery stats Signed-off-by: Sachin Kale <[email protected]> (cherry picked from commit e12a5b9) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
gbbafna
pushed a commit
that referenced
this pull request
Apr 4, 2023
…gments from remote store (#6825) (#6976) * Use existing StatsDirectoryWrapper to record recovery stats (cherry picked from commit e12a5b9) Signed-off-by: Sachin Kale <[email protected]>
mitrofmep
pushed a commit
to mitrofmep/OpenSearch
that referenced
this pull request
Apr 5, 2023
…gments from remote store (opensearch-project#6825) * Use existing StatsDirectoryWrapper to record recovery stats Signed-off-by: Sachin Kale <[email protected]> Signed-off-by: Valentin Mitrofanov <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
_cat/recoveryAPI provide details of progress of recovery. This helps in getting insight especially when shards are bigger in size._cat/recovery.Issues Resolved
_cat/recoveryshould provide progress of remote segment download during recovery #6821Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.