Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

config: single-node cluster shows all ranges as under-replicated #21429

Closed
nvanbenschoten opened this issue Jan 12, 2018 · 2 comments · Fixed by #28495
Closed

config: single-node cluster shows all ranges as under-replicated #21429

nvanbenschoten opened this issue Jan 12, 2018 · 2 comments · Fixed by #28495
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@nvanbenschoten
Copy link
Member

I assume this topic has a longer history than I'm aware of. Mainly opening to discuss.

At the moment, a single-node cluster is initialized with a zone-config that defaults to 3-way replication. As a result of this, we mark all ranges in the cluster as "under-replicated".

screen shot 2018-01-12 at 6 06 57 pm

We also log various messages that call this out, like:

E180110 04:34:11.439278 415 storage/queue.go:738  [n1,replicate] 221 replicas failing with "0 of 0 stores with all attributes matching []; likely not enough nodes in cluster"

A user can modify the zone-configs to specify a replication factor of 1, which should avoid this issue, but that it's the easiest thing to do. Is this the experience we want to require of users who want just a single-node cluster? I think a strong argument can be made for "yes", because we want to make sure any misconfigurations that prevent up-replication are called out when a multi-node cluster is desired.
@bdarnell
Copy link
Contributor

Historically, we thought a once-every-ten-minutes log message was reasonable for this, but that was before "under-replicated ranges" was displayed so prominently in the UI.

I've been in favor of treating single-node clusters as more distinct from multi-node clusters. For example, we could have a cockroach start-single-node command that would set the target replication factor to 1, and maybe also give other parameters different defaults (--host=localhost, for example). Then we could make --join and the init command mandatory for the multi-node case. Then instead of extra steps in our "start a single-node cluster" docs, we'd have a page on "upgrading a single-node cluster to multi-node" that would have you set num_replicas back.

@bdarnell bdarnell mentioned this issue Mar 21, 2018
Closed
@petermattis petermattis changed the title cfg: single-node cluster shows all ranges as under-replicated config: single-node cluster shows all ranges as under-replicated Jul 21, 2018
@petermattis petermattis added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-cli labels Jul 21, 2018
@knz knz mentioned this issue Aug 11, 2018
Merged
@bdarnell
Copy link
Contributor

This was fixed a while back as a part of adaptive zone configs.

craig bot pushed a commit that referenced this issue Aug 2, 2019
@knz
Merge #28495
175c5ad 
28495: cli: new command 'start-single-node' r=knz a=knz

Fixes #24118.
Fixes #21429.

Suggested/requested by @bdarnell:

> There are currently two ways to initialize a cluster: Starting the
  first node without a --join flag, or starting all nodes with --join
  flags and running a separate cockroach init command. This redundancy
  is confusing (our docs use both methods without explaining the
  difference) and it's easy to accidentally initialize a new cluster
  by restarting your first node with the wrong flags. (This could have
  catastrophic consequences in 1.1 because data from the two clusters
  could get mixed. We've added safeguards against this in 2.0 but it's
  still an easy way to break things.)
> I think that we should deprecate this implicit initialization mode
  and make the --join flag mandatory when starting a node (and
  therefore the init command would be mandatory too). To avoid adding
  too much complexity to single-node clusters, I propose a new
  `cockroach start-single-node` command to perform the start and init
  combination that is today implicit in the absence of a --join flag.
> This would have a side benefit that we would know when the user
  intends to keep the cluster as a single node instead of adding more
  later. The `start-single-node` command could therefore set the
  replication factor to 1 to disable the "underreplicated ranges"
  warning.

This patch implements the new `start-single-node` command as follows:

- the new command does not accept `--join`.
- it attempts to set the replication factor to 1 upon starting up.

Meanwhile `cockroach start` is changed to produce a deprecation
warning if `--join` is not specified.

Release note (cli change): a new command `cockroach start-single-node`
is introduced to start single-node clusters with replication disabled.

Release note (cli change): using `cockroach start` without `--join` is
now deprecated and this mode of execution will be removed in a later
version of CockroachDB. Consider using `cockroach start-single-node`
instead or combine `cockroach start` with `cockroach init`.

Co-authored-by: Raphael 'kena' Poss <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cli: new command 'start-single-node' knz/cockroach
3 participants
@bdarnell @nvanbenschoten @petermattis