-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Don't bootstrap when bootstrap_expect
is zero
#9672
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great find and the test fills an important gap. Don't forget the changelog entry.
nomad/serf_test.go
Outdated
time.Sleep(500 * time.Millisecond) | ||
|
||
bootstrapped := atomic.LoadInt32(&s1.config.Bootstrapped) | ||
require.Zero(t, bootstrapped, "expecting non-bootstrapped servers") | ||
|
||
p, _ := s1.numPeers() | ||
require.Zero(t, p, "number of peers in Raft") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be put in a WaitForResult instead of using a sleep? If not please comment on the sleep so it's clear why sleeping is the preferred approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's tricky, as we are testing for a negative in a concurrent environment: the server starts in non-bootstrap mode with 0 peers, and we want to test that the server remains so even it registers itself into serf. Without time delay, the test may pass just because the server hasn't setup raft yet. The sleep doesn't completely rule out that possibility, but makes it less likely.
I'll add a note.
Don't bootstrap when `bootstrap_expect` is zero
I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions. |
When
bootstrap_expect
is 0, the Nomad server shouldn't attempt to bootstrap and elect itself as the leader. This is the behavior of Consul, the previous Nomad behavior, as well as intended behaviod.The behavior was accidentally changed in https://github.com/hashicorp/nomad/pull/7252/files#diff-adc7208ec155586c7694004df7060da8916517d28041e0660d171d73bfdfee47L86-R86 . Prior to that, Nomad only attempted to bootstrap if BootstrapExpect is non-zero, while the change made it such that bootstrapping is always attempted regardless of bootstrap expect value.
Added a failing test in https://app.circleci.com/pipelines/github/hashicorp/nomad/13931/workflows/4f71cf9d-5617-4ce1-b92c-749cfddf05f2/jobs/127419 .