CCR: Following primary should process NoOps once #34408
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a follow-up for elastic#34288 (comment). Relates elastic#34288
dnhatn
added
>non-issue
:Distributed Indexing/CCR
|
Pinging @elastic/es-distributed |
bleskes
approved these changes
Oct 19, 2018
| @@ -2338,7 +2358,7 @@ public void waitForOpsToComplete(long seqNo) throws InterruptedException { | |||
| */ | |||
| protected final boolean hasBeenProcessedBefore(Operation op) { | |||
| assert op.seqNo() != SequenceNumbers.UNASSIGNED_SEQ_NO : "operation is not assigned seq_no"; | |||
| assert versionMap.assertKeyedLockHeldByCurrentThread(op.uid().bytes()); | |||
| assert noOpKeyedLock.isHeldByCurrentThread(op.seqNo()) || versionMap.assertKeyedLockHeldByCurrentThread(op.uid().bytes()); | |||
There was a problem hiding this comment.
can we condition this on the op type?
| protected Optional<Exception> preFlightCheckForNoOp(NoOp noOp) { | ||
| if (noOp.origin() == Operation.Origin.PRIMARY && hasBeenProcessedBefore(noOp)) { | ||
| // See the comment in #indexingStrategyForOperation for the explanation why we can safely skip this operation. | ||
| return Optional.of(new AlreadyProcessedFollowingEngineException(shardId, noOp.seqNo())); |
There was a problem hiding this comment.
Note that this will require changing the other PR to accept no opes with seq > gcp having no term and replicating them with current term.
bleskes
reviewed
Oct 19, 2018
| protected Optional<Exception> preFlightCheckForNoOp(NoOp noOp) throws IOException { | ||
| if (noOp.origin() == Operation.Origin.PRIMARY && hasBeenProcessedBefore(noOp)) { | ||
| // See the comment in #indexingStrategyForOperation for the explanation why we can safely skip this operation. | ||
| final OptionalLong existingTerm = lookupPrimaryTerm(noOp.seqNo()); |
There was a problem hiding this comment.
random thought - I wonder if we should load the operation under assertion code and check it's the same (this goes for all duplicate ops).
There was a problem hiding this comment.
Do you mean the existing operation should equal the processing operation except for the primary term?
There was a problem hiding this comment.
++. I'll make it in a follow-up after this PR.
Clear the injected exception after the indexing
|
Thanks @bleskes. |
dnhatn
added a commit
that referenced
this pull request
Oct 21, 2018
kcm
pushed a commit
that referenced
this pull request
Oct 30, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a follow-up for #34288 (comment).
Relates #34288