Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache DiscoveryNode#trimTier Result #80179

Conversation

original-brownbear
Copy link
Member

No need to recompute this over and over. This is the most expensive part of
FilterAllocationDecider in benchmarking. Since we already cache the filter itself
on the IndexMetadata, we can just cache the trimmed version also and
FilterAllocationDecider effectively disappears from profiling.
Also, made the filter immutable (which should as a side-effect also make it faster
when iterating over it in the match method).

relates #77466

No need to recompute this over and over. This is the most expensive part of
`FilterAllocationDecider` in benchmarking. Since we already cache the filter itself
on the `IndexMetadata`, we can just cache the trimmed version also and
`FilterAllocationDecider` effectively disappears from profiling.
Also, made the filter immutable (which should as a side-effect also make it faster
when iterating over it in the match method).
@original-brownbear original-brownbear added >non-issue :Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) v8.0.0 v8.1.0 labels Nov 2, 2021
@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Nov 2, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok with me - I left a couple of nits.

DiscoveryNodeFilters(OpType opType, Map<String, String[]> filters) {
this.opType = opType;
this.filters = filters;
this.filters = Map.copyOf(filters);
this.trimmed = doTrim(this);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: could we avoid even calling this when constructing the trimmed filters? Seems unnecessary to walk the filters twice just to check that trimming actually worked.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually ... we can just avoid that walk altogether in 8 :) Missed that fact earlier, but we don't really need to walk the map in 8 because we're only filtering out a specific key (and null which I moved elsewhere but I think that filtering is unused ... just didn't want to change behavior here).
Simplified accordingly now

@@ -66,9 +65,13 @@ public static DiscoveryNodeFilters buildFromKeyValue(OpType opType, Map<String,

private final OpType opType;

@Nullable
private final DiscoveryNodeFilters trimmed;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming nit: would be good to mention that it's the tier preference that was trimmed - withoutTierPreferences or similar perhaps?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ sounds good, renamed

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@original-brownbear
Copy link
Member Author

Thanks David!

@original-brownbear original-brownbear merged commit 268b189 into elastic:master Nov 2, 2021
@original-brownbear original-brownbear deleted the faster-trim-tier-disco-node-filters branch November 2, 2021 15:44
@original-brownbear original-brownbear restored the faster-trim-tier-disco-node-filters branch April 18, 2023 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport pending :Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) >non-issue Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. v8.0.0-beta1 v8.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants