Move update and delete by query to use seq# for optimistic concurrency control #37857
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bleskes
added
>enhancement
v7.0.0
:Distributed Indexing/Distributed
|
Pinging @elastic/es-distributed |
ywelsch
suggested changes
Jan 28, 2019
| @@ -31,20 +32,20 @@ | |||
| * Implementation of delete-by-query using scrolling and bulk. | |||
| */ | |||
| public class AsyncDeleteByQueryAction extends AbstractAsyncBulkByScrollAction<DeleteByQueryRequest> { | |||
| private final boolean useSeqNoForOCC; | |||
There was a problem hiding this comment.
OCC is a bit too complicated to guess :) Perhaps useSeqNoForCAS
There was a problem hiding this comment.
You've changed it in the other class, but not here? 😢
| @@ -270,6 +294,14 @@ public BasicHit setRouting(String routing) { | |||
| this.routing = routing; | |||
| return this; | |||
| } | |||
|
|
|||
| public void setSeqNo(long seqNo) { | |||
There was a problem hiding this comment.
where is this ever called? I guess this needs to be plugged into RemoteResponseParsers? Does this mean we are missing some tests?
There was a problem hiding this comment.
That's a good one. It's actually not relevant when indexing from remote (seqnos are only for when update the same index). This class is called "BasicHit" with a java doc of "An implementation of {@linkplain Hit} that uses getters and setters.", so I felt I had to add this, although it's not used.
Your point actually made me realize I added partial parsing support which wasn't wired and isn't needed. I removed it.
ywelsch
approved these changes
Jan 29, 2019
| @@ -31,20 +32,20 @@ | |||
| * Implementation of delete-by-query using scrolling and bulk. | |||
| */ | |||
| public class AsyncDeleteByQueryAction extends AbstractAsyncBulkByScrollAction<DeleteByQueryRequest> { | |||
| private final boolean useSeqNoForOCC; | |||
There was a problem hiding this comment.
You've changed it in the other class, but not here? 😢
|
Thanks @ywelsch ! |
bleskes
added a commit
to bleskes/elasticsearch
that referenced
this pull request
Jan 31, 2019
bleskes
added a commit
to bleskes/elasticsearch
that referenced
this pull request
Feb 1, 2019
…y control (elastic#37857) The delete and update by query APIs both offer protection against overriding concurrent user changes to the documents they touch. They currently are using internal versioning. This PR changes that to rely on sequences numbers and primary terms. Relates elastic#37639 Relates elastic#36148 Relates elastic#10708
bleskes
added a commit
to bleskes/elasticsearch
that referenced
this pull request
Feb 1, 2019
After backporting elastic#37977, elastic#37857 and elastic#37872
bleskes
added a commit
that referenced
this pull request
Feb 1, 2019
bleskes
added a commit
that referenced
this pull request
Feb 1, 2019
…37872 (#38155) * Move update and delete by query to use seq# for optimistic concurrency control (#37857) The delete and update by query APIs both offer protection against overriding concurrent user changes to the documents they touch. They currently are using internal versioning. This PR changes that to rely on sequences numbers and primary terms. Relates #37639 Relates #36148 Relates #10708 * Add Seq# based optimistic concurrency control to UpdateRequest (#37872) The update request has a lesser known support for a one off update of a known document version. This PR adds an a seq# based alternative to power these operations. Relates #36148 Relates #10708 * Move watcher to use seq# and primary term for concurrency control (#37977) * Adapt minimum versions for seq# power operations After backporting #37977, #37857 and #37872
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Feb 1, 2019
…ersion * elastic/master: SnapshotShardsService Simplifications (elastic#38025) Default include_type_name to false in the yml test harness. (elastic#38058) Disable bwc preparing to backport of#37977, elastic#37857 and elastic#37872 (elastic#38126) Adding ml_settings entry to HLRC and Docs for deprecation_info (elastic#38118)
bleskes
added a commit
to bleskes/elasticsearch
that referenced
this pull request
Feb 1, 2019
…equests and UpdateRequest After backporting elastic#37977, elastic#37857 and elastic#37872
jasontedor
added a commit
to bleskes/elasticsearch
that referenced
this pull request
Feb 1, 2019
* elastic/master: AwaitsFix testAbortedSnapshotDuringInitDoesNotStart (elastic#38227) Preserve ILM operation mode when creating new lifecycles (elastic#38134) Enable trace log in FollowerFailOverIT (elastic#38148) SnapshotShardsService Simplifications (elastic#38025) Default include_type_name to false in the yml test harness. (elastic#38058) Disable bwc preparing to backport of#37977, elastic#37857 and elastic#37872 (elastic#38126)
jasontedor
added a commit
to gwbrown/elasticsearch
that referenced
this pull request
Feb 1, 2019
* elastic/master: (54 commits) Introduce retention leases versioning (elastic#37951) Correctly disable tests for FIPS JVMs (elastic#38214) AwaitsFix testAbortedSnapshotDuringInitDoesNotStart (elastic#38227) Preserve ILM operation mode when creating new lifecycles (elastic#38134) Enable trace log in FollowerFailOverIT (elastic#38148) SnapshotShardsService Simplifications (elastic#38025) Default include_type_name to false in the yml test harness. (elastic#38058) Disable bwc preparing to backport of#37977, elastic#37857 and elastic#37872 (elastic#38126) Adding ml_settings entry to HLRC and Docs for deprecation_info (elastic#38118) Replace awaitBusy with assertBusy in atLeastDocsIndexed (elastic#38190) Adjust SearchRequest version checks (elastic#38181) AwaitsFix testClientSucceedsWithVerificationDisabled (elastic#38213) Zen2ify RareClusterStateIT (elastic#38184) ML: Fix error race condition on stop _all datafeeds and close _all jobs (elastic#38113) AwaitsFix PUT mapping with _doc on an index that has types (elastic#38204) Allow built-in monitoring_user role to call GET _xpack API (elastic#38060) Update geo_shape docs to include unsupported features (elastic#38138) [ML] Remove "8" prefixes from file structure finder timestamp formats (elastic#38016) Disable bwc tests while backporting elastic#38104 (elastic#38182) Enable TLSv1.3 by default for JDKs with support (elastic#38103) ...
jasontedor
pushed a commit
that referenced
this pull request
Feb 2, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The delete and update by query APIs both offer protection against overriding concurrent user changes to the documents they touch. They currently are using internal versioning. This PR changes that to rely on sequences numbers and primary terms.
Relates #37639
Relates #36148
Relates #10708