Make PrimaryReplicaResyncer Fork to Generic Pool #69949
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reading ops from the translog snapshot must not run on the transport thread. When sending more than one batch of ops the listener (and thus `run`) would be invoked on the transport thread for all but the first batch of ops. => Forking to the generic pool like we do for sending ops during recovery.
original-brownbear
added
>bug
:Distributed Indexing/CRUD
elasticmachine
added
the
Team:Distributed (Obsolete)
|
Pinging @elastic/es-distributed (Team:Distributed) |
DaveCTurner
reviewed
Mar 4, 2021
| @@ -159,7 +160,8 @@ protected DiscoveryNode getDiscoveryNode(String id) { | |||
| private volatile ReplicationTargets replicationTargets; | |||
|
|
|||
| private final PrimaryReplicaSyncer primaryReplicaSyncer = new PrimaryReplicaSyncer( | |||
| new TaskManager(Settings.EMPTY, threadPool, Collections.emptySet()), | |||
| new MockTransport().createTransportService(Settings.EMPTY, threadPool, | |||
There was a problem hiding this comment.
Do we need to shut this (at least the threadpool) down cleanly?
There was a problem hiding this comment.
It's just using the fake transport so I don;t think the service needs any shutting down here. The threadPool is torn down via the parent IndexShardTestCase -> I think we're good?
There was a problem hiding this comment.
Oh right we already had a threadpool we're just using the transport service as a wrapper. Nvm.
DaveCTurner
approved these changes
Mar 4, 2021
| @@ -159,7 +160,8 @@ protected DiscoveryNode getDiscoveryNode(String id) { | |||
| private volatile ReplicationTargets replicationTargets; | |||
|
|
|||
| private final PrimaryReplicaSyncer primaryReplicaSyncer = new PrimaryReplicaSyncer( | |||
| new TaskManager(Settings.EMPTY, threadPool, Collections.emptySet()), | |||
| new MockTransport().createTransportService(Settings.EMPTY, threadPool, | |||
There was a problem hiding this comment.
Oh right we already had a threadpool we're just using the transport service as a wrapper. Nvm.
|
Thanks David! |
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Mar 4, 2021
Reading ops from the translog snapshot must not run on the transport thread. When sending more than one batch of ops the listener (and thus `run`) would be invoked on the transport thread for all but the first batch of ops. => Forking to the generic pool like we do for sending ops during recovery.
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Mar 4, 2021
Reading ops from the translog snapshot must not run on the transport thread. When sending more than one batch of ops the listener (and thus `run`) would be invoked on the transport thread for all but the first batch of ops. => Forking to the generic pool like we do for sending ops during recovery.
original-brownbear
added a commit
that referenced
this pull request
Mar 4, 2021
Reading ops from the translog snapshot must not run on the transport thread. When sending more than one batch of ops the listener (and thus `run`) would be invoked on the transport thread for all but the first batch of ops. => Forking to the generic pool like we do for sending ops during recovery.
original-brownbear
added a commit
that referenced
this pull request
Mar 4, 2021
Reading ops from the translog snapshot must not run on the transport thread. When sending more than one batch of ops the listener (and thus `run`) would be invoked on the transport thread for all but the first batch of ops. => Forking to the generic pool like we do for sending ops during recovery.
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this pull request
Mar 17, 2021
We assert that the snapshot isn't closed on a transport thread, but we close it without forking off the transport thread in case of a failure. With this commit we fork on failure too. Relates elastic#69949 Closes elastic#70407
DaveCTurner
added a commit
that referenced
this pull request
Mar 17, 2021
DaveCTurner
added a commit
that referenced
this pull request
Mar 17, 2021
DaveCTurner
added a commit
that referenced
this pull request
Mar 17, 2021
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Mar 30, 2021
We can have a race here where the closed check passes and then we concurrently to a shard close try to fail the shard also. Previously this was covered by the catch below the changed code that would just ignore the already-closed exception but with elastic#69949 we're now forking to the generic pool for this logic and thus have to handle the exception in the callback as well.
original-brownbear
added a commit
that referenced
this pull request
Mar 31, 2021
We can have a race here where the closed check passes and then we concurrently to a shard close try to fail the shard also. Previously this was covered by the catch below the changed code that would just ignore the already-closed exception but with #69949 we're now forking to the generic pool for this logic and thus have to handle the exception in the callback as well.
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Mar 31, 2021
We can have a race here where the closed check passes and then we concurrently to a shard close try to fail the shard also. Previously this was covered by the catch below the changed code that would just ignore the already-closed exception but with elastic#69949 we're now forking to the generic pool for this logic and thus have to handle the exception in the callback as well.
original-brownbear
added a commit
to original-brownbear/elasticsearch
that referenced
this pull request
Mar 31, 2021
We can have a race here where the closed check passes and then we concurrently to a shard close try to fail the shard also. Previously this was covered by the catch below the changed code that would just ignore the already-closed exception but with elastic#69949 we're now forking to the generic pool for this logic and thus have to handle the exception in the callback as well.
original-brownbear
added a commit
that referenced
this pull request
Mar 31, 2021
We can have a race here where the closed check passes and then we concurrently to a shard close try to fail the shard also. Previously this was covered by the catch below the changed code that would just ignore the already-closed exception but with #69949 we're now forking to the generic pool for this logic and thus have to handle the exception in the callback as well.
original-brownbear
added a commit
that referenced
this pull request
Mar 31, 2021
We can have a race here where the closed check passes and then we concurrently to a shard close try to fail the shard also. Previously this was covered by the catch below the changed code that would just ignore the already-closed exception but with #69949 we're now forking to the generic pool for this logic and thus have to handle the exception in the callback as well.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reading ops from the translog snapshot must not run on the transport thread.
When sending more than one batch of ops the listener (and thus
run) would beinvoked on the transport thread for all but the first batch of ops.
=> Forking to the generic pool like we do for sending ops during recovery.