Add index tracking to metastore. #1672
Merged
+153
−114
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds tracking of the index to the metastore so that data nodes will ignore replayed indexes when they are restarted. This does not include index persistence for the shards.
|
Nice. Assuming the server index read from the Metatstore is hen used in the right way, ship it. |
benbjohnson
added a commit
that referenced
this pull request
Feb 21, 2015
Add index tracking to metastore.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This pull request adds tracking of the index to the metastore so that data nodes will ignore replayed indexes when they are restarted. This does not include index persistence for the shards.
Changes
metastore.mustUpdate()takes anindexparameter which will automatically set the index on the metastore.Server.indexon startup and when the metastore is copied.applyfunctions and moved them to theServer.processor(). This removes double locking we were doing before.applyWriteSeries()to be processed separately from the other commands so we don't server lock around every series write.Caveats
This does not include persistence of shard indices although that's less of an issue since shard writes are currently idempotent. We'll need to add that in a separate PR.