Diagnostics reporting and global settings

[jseldess]

• Add page on diagnostic metrics sent to Cockroach Labs and how to opt out.
• Add page on how to view and change cluster-wide settings.
• Add note about cluster-wide settings to cockroach start docs.
• Update SQL reference docs concerning cluster-wide settings.
• Add new pages to the sidenav.

Partly address #1277.

---

This change is 

[dt]

Reporting page looks great!

I might suggest for the unfinished configs page that we skip listing/describing all the individual settings for now and just call out the SHOW ALL syntax as a way to list the ones supported in a given version.

We should also mention that the settings are shared between all nodes and thus it is usually a good idea not to adjust a new setting during a cluster upgrade, while older modes may not handle it -- we attempt to gracefully ignore unknown settings written by other nodes or even unrecognized values for known settings (e.g. If a newer node uses adds an optionC to an A-or-B setting, that node would allow setting it to C), but it is still recommended to ensure all nodes are upgraded before changing new settings.

[jseldess]

Thanks, @dt. I removed explanations of current global settings, and I added some callouts before you change a global setting. PTAL.

Will move on to the SHOW and SET reference docs now.

[dianasaur323]

I'm on the side that we might be overcompensating in terms of the detail of information we are sharing. I'm trying to find some other examples of how people do it, but I can't seem to find a good example.

---

Review status: 0 of 3 files reviewed at latest revision, all discussions resolved.

---

Comments from Reviewable

[jseldess]

Thanks, @dianasaur323. I personally feel that we're not sharing anything alarming; rather, the details try to clarify that there's no need for alarm at all. But let's get @bdarnell's opinion here.

[sploiselle]

For being a transparent, open-source project, I think this approach aligns with our ethics and does so really tastefully. Really great work on this, @jseldress.

---

Review status: 0 of 3 files reviewed at latest revision, 7 unresolved discussions.

---

global-configurations.md, line 9 at r1 (raw file):

In CockroachDB, there are two levels of configuration:

- 	**Node-level configurations** apply to a single node of a CockroachDB cluster and define, for example, the interfaces for communicating with other nodes and the percent of system memory allocated to key-value operations performed by the node. These settings are defined when a node is started with `cockroach start` and cannot be changed without stopping and restarting the node.

Because node-level configurations do not apply globally, I would change the architecture of this a little bit to align the content to the title. Instead of talking about the two levels of configuration, I would address cluster-wide configurations primarily and only make a note of the node-level configurations.

Alternatively, you could change the title of the page to something like "Node- and Cluster-Level Configurations."

Also, re: whatever we end up doing to discuss node-level configs, I would also clarify that the settings are defined by the flags passed to the node through cockroach start, but that is maybe too pedantic.

---

global-configurations.md, line 11 at r1 (raw file):

- 	**Node-level configurations** apply to a single node of a CockroachDB cluster and define, for example, the interfaces for communicating with other nodes and the percent of system memory allocated to key-value operations performed by the node. These settings are defined when a node is started with `cockroach start` and cannot be changed without stopping and restarting the node.

- 	**Cluster-wide configurations** apply to all nodes of CockroachDB cluster and define, for example, whether or not to share diagnostic details with Cockroach Labs and advanced options for debugging and cluster tuning. These settings are defined via SQL `SET` statements and therefore can be updated anytime after a cluster has been started.

"via SQL SET CLUSTER statements"?

---

diagnostic-reporting.md, line 9 at r1 (raw file):

By default, each node of a CockroachDB cluster shares diagnostic details with Cockroach Labs on a daily basis. These details greatly help us understand and improve how the system behaves in real-world scenarios, and they are completely scrubbed of identifiable information.

This page explains the details that get shared and how to opt out of sharing.

Maybe add a link to the Admin UI doc pages here inline or in a note that says something to the effect of, "For information about your cluster's diagnostics, use our Admin UI." Think it would be easy to mistake this page's title for Admin UI-type info.

---

diagnostic-reporting.md, line 15 at r1 (raw file):

## What Gets Shared

When diagnostic reporting is on, each node of a CockroachDB cluster shares storage and table schema details with Cockroach Labs on a daily basis.

"shares anonymized storage and table schema details"

---

diagnostic-reporting.md, line 24 at r1 (raw file):

-------|------------
Node ID | The ID of the node.
Store ID | The ID of each store on the node.

My slight confusion/apprehension here is if these values uniquely identify my cluster. The Table description below expresses that the ID is just an internal ID; is this ID the same kind of thing?

---

diagnostic-reporting.md, line 189 at r1 (raw file):

~~~

## How to Opt Out

nit: Would prefer a title like "Opt Out of Diagnostics Reporting"

---

diagnostic-reporting.md, line 197 at r1 (raw file):

~~~

Note that this change will not be instantaneous, as it must be propigated to other nodes in the cluster. If you want to make sure that no diagnostic details are shared, you can set the environment variable `COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING=true` before starting the first node of the cluster. Note that this works only when set before starting the first node of the cluster. Once the cluster is running, you need to use the `SET` method described above.

s/propigated/propagated

---

Comments from Reviewable

[jseldess]

Review status: 0 of 5 files reviewed at latest revision, 7 unresolved discussions, some commit checks failed.

---

diagnostics-reporting.md, line 9 at r1 (raw file):

Previously, sploiselle (Sean Loiselle) wrote…

Maybe add a link to the Admin UI doc pages here inline or in a note that says something to the effect of, "For information about your cluster's diagnostics, use our Admin UI." Think it would be easy to mistake this page's title for Admin UI-type info.

Done.

---

diagnostics-reporting.md, line 15 at r1 (raw file):

Previously, sploiselle (Sean Loiselle) wrote…

"shares anonymized storage and table schema details"

Done.

---

diagnostics-reporting.md, line 24 at r1 (raw file):

Previously, sploiselle (Sean Loiselle) wrote…

My slight confusion/apprehension here is if these values uniquely identify my cluster. The Table description below expresses that the ID is just an internal ID; is this ID the same kind of thing?

Yeah, by internal, I just mean that CockroachDB assigns the value rather than a user/client. I've added that, but let me know if you think I should add another sentence.

---

diagnostics-reporting.md, line 189 at r1 (raw file):

Previously, sploiselle (Sean Loiselle) wrote…

nit: Would prefer a title like "Opt Out of Diagnostics Reporting"

Done.

---

diagnostics-reporting.md, line 197 at r1 (raw file):

Previously, sploiselle (Sean Loiselle) wrote…

s/propigated/propagated

Done. I really need to use an automated spellchecker!

---

global-configurations.md, line 9 at r1 (raw file):

Previously, sploiselle (Sean Loiselle) wrote…

Because node-level configurations do not apply globally, I would change the architecture of this a little bit to align the content to the title. Instead of talking about the two levels of configuration, I would address cluster-wide configurations primarily and only make a note of the node-level configurations.

Alternatively, you could change the title of the page to something like "Node- and Cluster-Level Configurations."

Also, re: whatever we end up doing to discuss node-level configs, I would also clarify that the settings are defined by the flags passed to the node through cockroach start, but that is maybe too pedantic.

Done. Great feedback.

---

global-configurations.md, line 11 at r1 (raw file):

Previously, sploiselle (Sean Loiselle) wrote…

"via SQL SET CLUSTER statements"?

Done.

---

Comments from Reviewable

[sploiselle]

really great work

---

Reviewed 4 of 4 files at r2, 2 of 2 files at r3.
Review status: all files reviewed at latest revision, 6 unresolved discussions, some commit checks failed.

---

Comments from Reviewable

[cockroach-teamcity]

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/504f082fe0d12f2971e03e8a4a5109fe24f4f59c/

[cockroach-teamcity]

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/d976595f1d0f1be01b3841bda6c17627ec03134f/

[knz]

Reviewed 2 of 4 files at r2, 1 of 2 files at r3, 3 of 3 files at r4.
Review status: all files reviewed at latest revision, 9 unresolved discussions.

---

diagnostics-reporting.md, line 66 at r4 (raw file):

Each node of a CockroachDB cluster shares the following schema details for each table stored on the node:

{{site.data.alerts.callout_info}}No actual table data is shared, just metadata about the schema of tables, and all name and other string values are scrubbed and replaced with underscores.{{site.data.alerts.end}}

I would place a period between "about the schema of tables" and "All names and ..."

---

global-configurations.md, line 11 at r1 (raw file):

Previously, jseldess wrote…

Done.

Mention also that only the root user can modify settings.
(And perhaps highlight that's one more reason to create users for client apps in a production environment, because client apps logged as root could change these and that would be unsafe.)

---

global-configurations.md, line 69 at r4 (raw file):

Before changing a global configuration, please note the following:

- 	Many global configurations are intended for tuning CockroachDB internals. Before changing these settings, it's best to [discuss your goals with CockroachDB](https://gitter.im/cockroachdb/cockroach); otherwise, you use them at your own risk.

I think the responsibility disclaimer is too weak with this phrasing.
Right now I could read that "As long as I mentioned it on gitter, even though it was on sunday at night and nobody noticed, I did my share of telling and thus CRL is responsible for the consequences."

Perhaps invert the sentence: "Use this configuration mechanism at your own risk. If you think you need to tune these knobs, we encourage you to first [discuss your goals ...]. "

---

_data/sidebar_doc.yml, line 205 at r4 (raw file):

                  url: /select.html

                - title: <code>SET DATABASE</code>

SET CLUSTER SETTING is missing here (and SHOW [ALL] CLUSTER SETTING) below

---

Comments from Reviewable

[jseldess]

Review status: all files reviewed at latest revision, 9 unresolved discussions.

---

diagnostics-reporting.md, line 66 at r4 (raw file):

Previously, knz (kena) wrote…

I would place a period between "about the schema of tables" and "All names and ..."

Done.

---

global-configurations.md, line 11 at r1 (raw file):

Previously, knz (kena) wrote…

Mention also that only the root user can modify settings.
(And perhaps highlight that's one more reason to create users for client apps in a production environment, because client apps logged as root could change these and that would be unsafe.)

I added the detail about SET CLUSTER SETTING being limited to the root user. Thanks for pointing that out! I didn't add the detail about non-root users for client apps. It's a very good point, but the guidance is broader than this feature, so I want to find the right place for it. Opened this issue: #1342

---

global-configurations.md, line 69 at r4 (raw file):

Previously, knz (kena) wrote…

I think the responsibility disclaimer is too weak with this phrasing.
Right now I could read that "As long as I mentioned it on gitter, even though it was on sunday at night and nobody noticed, I did my share of telling and thus CRL is responsible for the consequences."

Perhaps invert the sentence: "Use this configuration mechanism at your own risk. If you think you need to tune these knobs, we encourage you to first [discuss your goals ...]. "

Strengthened the language, styled it as a warning, and listed it first.

---

_data/sidebar_doc.yml, line 205 at r4 (raw file):

Previously, knz (kena) wrote…

SET CLUSTER SETTING is missing here (and SHOW [ALL] CLUSTER SETTING) below

I didn't create stand-alone pages for these statements but rather covered them in the global-configurations.md page. Though I did list them on sql-statements.md, I can't add them to the sidenav as you ask due to a quirk in the way the sidenav behaves: If multiple items in the sidenav match the current url, all sections containing those items are expanded, which is just bad user experience. This is definitely something we should fix at some point, since it's natural to want to link to the same page from multiple parts of a navigation sometimes.

---

Comments from Reviewable

[bdarnell]

LGTM. I agree with being detailed about what's being collected, and especially with being explicit about the scrubbing being done.

---

Reviewed 1 of 4 files at r2, 1 of 2 files at r5.
Review status: 4 of 5 files reviewed at latest revision, 15 unresolved discussions.

---

diagnostics-reporting.md, line 17 at r5 (raw file):

## What Gets Shared

When diagnostics reporting is on, each node of a CockroachDB cluster shares anonymized storage and table schema details with Cockroach Labs on a daily basis.

How do we want to handle the fact that this set will change over time? In particular we're close to adding details about query skeletons (I think this will be going in for 1.0). It's great that we're specific about what gets shared, but we want to be clear that we may change this in future releases.

---

diagnostics-reporting.md, line 33 at r5 (raw file):

#### Example

This JSON example shows what storage details look like when sent to Cockroach Labs, in this case for a node with two stores:

I think I'd drop all the json examples; they're fairly redundant and make the page much more verbose. Or we could collapse them somehow so you can still get to them if you want, but I don't know if it's worth keeping them around (and up to date) if they won't be shown by default (or maybe it is, since it would be nice to show the effects of the scrubbing, especially for the query stats)

On the other hand, it would be nice if we had a link in the admin UI to show a copy of all the uploaded information for your cluster.

---

diagnostics-reporting.md, line 64 at r5 (raw file):

### SQL Schemas

Each node of a CockroachDB cluster shares the following schema details for each table stored on the node:

We may want to use the word "structure" here instead of "schema", to emphasize that we're only sending the tables in a skeletal form.

---

diagnostics-reporting.md, line 66 at r5 (raw file):

Each node of a CockroachDB cluster shares the following schema details for each table stored on the node:

{{site.data.alerts.callout_info}}No actual table data is shared, just metadata about the schema of tables. All name and other string values are scrubbed and replaced with underscores.{{site.data.alerts.end}}

"No actual table data or table/column names are shared..."

---

global-configurations.md, line 2 at r5 (raw file):

---
title: Global Configurations

Maybe call this "cluster settings" since that's what the commands say?

---

global-configurations.md, line 28 at r5 (raw file):

-------|------| -----------
`diagnostics.reporting.enabled` | Boolean | Whether or not nodes sends diagnostic details to CockroachDB. For more details, see [Diagnostics Reporting](diagnostics-reporting.html).<br><br>**Possible values:** `true` or `false`<br>**Default:** `true`
`enterprise.enabled` | Boolean | Whether or not Enterprise-grade features covered under the CockroachDB Community License (CCL) are enabled. For pricing and other details, please contact [CockroachDB Sales](https://www.cockroachlabs.com/pricing/sales/).

Remove this. We're hiding the enterprise.enabled setting for 1.0.

---

Comments from Reviewable

[jseldess]

Review status: 0 of 5 files reviewed at latest revision, 15 unresolved discussions.

---

diagnostics-reporting.md, line 17 at r5 (raw file):

Previously, bdarnell (Ben Darnell) wrote…

How do we want to handle the fact that this set will change over time? In particular we're close to adding details about query skeletons (I think this will be going in for 1.0). It's great that we're specific about what gets shared, but we want to be clear that we may change this in future releases.

I've added a note that these details may change in the future. I think we should just strive to update this page when we do change or add things.

---

diagnostics-reporting.md, line 33 at r5 (raw file):

Previously, bdarnell (Ben Darnell) wrote…

I think I'd drop all the json examples; they're fairly redundant and make the page much more verbose. Or we could collapse them somehow so you can still get to them if you want, but I don't know if it's worth keeping them around (and up to date) if they won't be shown by default (or maybe it is, since it would be nice to show the effects of the scrubbing, especially for the query stats)

On the other hand, it would be nice if we had a link in the admin UI to show a copy of all the uploaded information for your cluster.

Yeah, I like providing the JSON as proof that we're scrubbing strings. I agree that they make the page more verbose, though, especially the schema example, so I've truncated most of the it away and called out that it's just an excerpt of what table structure data looks like when it gets sent.

---

diagnostics-reporting.md, line 64 at r5 (raw file):

Previously, bdarnell (Ben Darnell) wrote…

We may want to use the word "structure" here instead of "schema", to emphasize that we're only sending the tables in a skeletal form.

Done.

---

diagnostics-reporting.md, line 66 at r5 (raw file):

Previously, bdarnell (Ben Darnell) wrote…

"No actual table data or table/column names are shared..."

Done.

---

global-configurations.md, line 2 at r5 (raw file):

Previously, bdarnell (Ben Darnell) wrote…

Maybe call this "cluster settings" since that's what the commands say?

Done.

---

global-configurations.md, line 28 at r5 (raw file):

Previously, bdarnell (Ben Darnell) wrote…

Remove this. We're hiding the enterprise.enabled setting for 1.0.

Done. This section is commented out for now, but might as well remove this setting now.

---

Comments from Reviewable

[cockroach-teamcity]

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/b65887ea447420cad0d333d3bd3773d99ed6a3db/

[cockroach-teamcity]

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/d77b6bd57d6ce0e4fc83ce707477b745513cfb42/