Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consul: allow stale namespace results #12953

Merged
merged 1 commit into from
Aug 26, 2022
Merged

Conversation

schmichael
Copy link
Member

Nomad reconciles services it expects to be registered in Consul with
what is actually registered in the local Consul agent. This is necessary
to prevent leaking service registrations if Nomad crashes at certain
points (or if there are bugs).

When Consul has namespaces enabled, we must iterate over each available
namespace to be sure no services were leaked into non-default
namespaces.

Since this reconciliation happens often, there's no need to require
results from the Consul leader server. In large clusters this creates
far more load than the "freshness" of the response is worth.

Therefore this patch switches the request to AllowStale=true

@schmichael schmichael added backport/1.2.x backport to 1.1.x release line backport/1.3.x backport to 1.3.x release line labels May 11, 2022
@schmichael schmichael requested review from shoenig and lgfa29 May 11, 2022 20:45
@schmichael schmichael force-pushed the f-allow-stale-consul-ns branch from 7bf77c5 to 323132b Compare May 11, 2022 20:46
@shoenig
Copy link
Member

shoenig commented May 12, 2022

@schmichael is this ready to review or should still be in draft? If it's ready, can you rebase with main to fix the GHA runners?

@shoenig
Copy link
Member

shoenig commented May 13, 2022

This change seems like a great improvement. We could probably also take it up a notch by caching the namespace list results in NamespaceClient.List, like we already for for caching the SKU/enabled result.

@schmichael schmichael marked this pull request as ready for review May 16, 2022 23:24
@schmichael
Copy link
Member Author

@shoenig recorded that #13035 for future work.

Copy link
Member

@shoenig shoenig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@schmichael schmichael added this to the 1.4.0 milestone Aug 26, 2022
Nomad reconciles services it expects to be registered in Consul with
what is actually registered in the local Consul agent. This is necessary
to prevent leaking service registrations if Nomad crashes at certain
points (or if there are bugs).

When Consul has namespaces enabled, we must iterate over each available
namespace to be sure no services were leaked into non-default
namespaces.

Since this reconciliation happens often, there's no need to require
results from the Consul leader server. In large clusters this creates
far more load than the "freshness" of the response is worth.

Therefore this patch switches the request to AllowStale=true
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 25, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backport/1.2.x backport to 1.1.x release line backport/1.3.x backport to 1.3.x release line
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants