Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vault: detect namespace change in config reload #14298

Merged
merged 3 commits into from
Aug 24, 2022
Merged

Conversation

tgross
Copy link
Member

@tgross tgross commented Aug 24, 2022

The namespace field was not included in the equality check between old and new
Vault configurations, which meant that a Vault config change that only changed
the namespace would not be detected as a change and the clients would not be
reloaded.

Also, the comparison for boolean fields such as enabled and
allow_unauthenticated was on the pointer and not the value of that pointer,
which results in spurious reloads in real config reload that is easily missed in
typical test scenarios.

Includes a minor refactor of the order of fields for Copy and Merge to match
the struct fields in hopes it makes it harder to make this mistake in the
future, as well as additional test coverage.


This comes out of #14233 but doesn't fix it as far as I can tell.

@tgross tgross force-pushed the b-vault-unsupported-path branch from f92a2db to f305849 Compare August 24, 2022 19:01
@tgross tgross marked this pull request as ready for review August 24, 2022 19:02
@tgross tgross added type/bug theme/vault backport/1.1.x backport to 1.1.x release line backport/1.2.x backport to 1.1.x release line backport/1.3.x backport to 1.3.x release line labels Aug 24, 2022
@tgross tgross added this to the 1.3.4 milestone Aug 24, 2022
The `namespace` field was not included in the equality check between old and new
Vault configurations, which meant that a Vault config change that only changed
the namespace would not be detected as a change and the clients would not be
reloaded.

Also, the comparison for boolean fields such as `enabled` and
`allow_unauthenticated` was on the pointer and not the value of that pointer,
which results in spurious reloads in real config reload that is easily missed in
typical test scenarios.

Includes a minor refactor of the order of fields for `Copy` and `Merge` to match
the struct fields in hopes it makes it harder to make this mistake in the
future, as well as additional test coverage.
@tgross tgross force-pushed the b-vault-unsupported-path branch from f305849 to b2c2ab5 Compare August 24, 2022 19:15
Copy link
Contributor

@lgfa29 lgfa29 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments on the test, but LGTM!

nomad/structs/config/vault_test.go Outdated Show resolved Hide resolved
Comment on lines 1108 to 1114
newConfig := &Config{
TLSConfig: &config.TLSConfig{},
Vault: &config.VaultConfig{
Enabled: pointer.Of(true),
Token: "vault-token",
Namespace: "vault-namespace",
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is newConfig supposed to be different from agentConfig? It think you would want to test if newConfig was applied after a Reload?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh good call. And I should be checking agent.GetConfig().Vault here too to get the updated config.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well as it turns out, reloading the config doesn't set the server config we keep in the server struct. That still holds on to the old version of the config as far as I can tell, because we're never supposed to read from it at runtime. That seems like a bug, tbh, but unrelated and there's a lot of threads to pull on that.

We do overwrite the VaultConfig that we write into the server's VaultClient, but I can't get to the one in the agent in this test. I've added an accessor method so that we can access it in the nomad/server_test.go test at least.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 9692586

Comment on lines +249 to +253
if c.TLSSkipVerify == nil || b.TLSSkipVerify == nil {
if c.TLSSkipVerify != b.TLSSkipVerify {
return false
}
} else if *c.TLSSkipVerify != *b.TLSSkipVerify {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was thinking, maybe we could have pointer.Compare similar to (and replacing) helper.CompareTimePtrs - caveat being it should only be used on basic types (not on structs with also pointer fields)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea, lemme take a crack at that.

Copy link
Member

@shoenig shoenig Aug 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

caveat being it should only be used on basic types

what am I thinking, we can enforce that with a type parameter constraint!

Copy link
Member

@shoenig shoenig Aug 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pretty sure this is effectively what we'd want

// Compare returns whether a and b contain the same value.
func Compare[A constraints.Ordered](a, b *A) bool {
// but also check nil
	return *a == *b
}```

Copy link
Member

@schmichael schmichael Aug 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, something like this? https://go.dev/play/p/nE1aQVTzsuz

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the problem is I want to prevent comparisons like this:

https://go.dev/play/p/Lh26c-YWmBd

using constraints.Ordered as a constraint prevents comparing pointers of anything but underlying Numeric/String types which are safe

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hrm, I seem to be stuck there.

type primitivePointer interface {
	*bool | *string | *int // etc, we can add the rest here
}

// Compare is an equality check for pointers that point to primitives.
func Compare[A primitivePointer](a, b A) bool {
	if a == nil || b == nil {
		return a == b
	}

	return *a == *b
}

Gives me:

# github.com/hashicorp/nomad/helper/pointer
helper/pointer/pointer.go:28:10: invalid operation: pointers of a (variable of type A constrained by primitivePointer) must have identical base types
helper/pointer/pointer.go:28:16: invalid operation: pointers of b (variable of type A constrained by primitivePointer) must have identical base types

For now I'm going to merge the bugfix and then we'll come up with some nicer comparison functions in a separate PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I posted that without seeing the other solutions. In any case, probably better for another PR.

@tgross tgross merged commit e886d5d into main Aug 24, 2022
@tgross tgross deleted the b-vault-unsupported-path branch August 24, 2022 21:03
tgross added a commit that referenced this pull request Aug 24, 2022
The `namespace` field was not included in the equality check between old and new
Vault configurations, which meant that a Vault config change that only changed
the namespace would not be detected as a change and the clients would not be
reloaded.

Also, the comparison for boolean fields such as `enabled` and
`allow_unauthenticated` was on the pointer and not the value of that pointer,
which results in spurious reloads in real config reload that is easily missed in
typical test scenarios.

Includes a minor refactor of the order of fields for `Copy` and `Merge` to match
the struct fields in hopes it makes it harder to make this mistake in the
future, as well as additional test coverage.
tgross added a commit that referenced this pull request Aug 24, 2022
The `namespace` field was not included in the equality check between old and new
Vault configurations, which meant that a Vault config change that only changed
the namespace would not be detected as a change and the clients would not be
reloaded.

Also, the comparison for boolean fields such as `enabled` and
`allow_unauthenticated` was on the pointer and not the value of that pointer,
which results in spurious reloads in real config reload that is easily missed in
typical test scenarios.

Includes a minor refactor of the order of fields for `Copy` and `Merge` to match
the struct fields in hopes it makes it harder to make this mistake in the
future, as well as additional test coverage.

Co-authored-by: Tim Gross <[email protected]>
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backport/1.1.x backport to 1.1.x release line backport/1.2.x backport to 1.1.x release line backport/1.3.x backport to 1.3.x release line theme/vault type/bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants