Skip to content

Commit

Permalink
sql: add system.span_configurations for kv span configs
Browse files Browse the repository at this point in the history
Part of cockroachdb#67679. It's not wired up to anything yet; we'll later use this
system-tenant only table to store KV span configs.

Release note (sql change): We've added a `system.span_configurations`
table. This will later be used to store authoritative span configs that
KV has decided to apply.
  • Loading branch information
irfansharif committed Aug 24, 2021
1 parent 5ba04b9 commit 4054d34
Show file tree
Hide file tree
Showing 33 changed files with 280 additions and 106 deletions.
2 changes: 1 addition & 1 deletion docs/generated/settings/settings-for-tenants.txt
Original file line number Diff line number Diff line change
Expand Up @@ -153,4 +153,4 @@ trace.datadog.project string CockroachDB the project under which traces will be
trace.debug.enable boolean false if set, traces for recent requests can be seen at https://<ui>/debug/requests
trace.lightstep.token string if set, traces go to Lightstep using this token
trace.zipkin.collector string if set, traces go to the given Zipkin instance (example: '127.0.0.1:9411'). Only one tracer can be configured at a time.
version version 21.1-148 set the active cluster version in the format '<major>.<minor>'
version version 21.1-150 set the active cluster version in the format '<major>.<minor>'
2 changes: 1 addition & 1 deletion docs/generated/settings/settings.html
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,6 @@
<tr><td><code>trace.debug.enable</code></td><td>boolean</td><td><code>false</code></td><td>if set, traces for recent requests can be seen at https://<ui>/debug/requests</td></tr>
<tr><td><code>trace.lightstep.token</code></td><td>string</td><td><code></code></td><td>if set, traces go to Lightstep using this token</td></tr>
<tr><td><code>trace.zipkin.collector</code></td><td>string</td><td><code></code></td><td>if set, traces go to the given Zipkin instance (example: '127.0.0.1:9411'). Only one tracer can be configured at a time.</td></tr>
<tr><td><code>version</code></td><td>version</td><td><code>21.1-148</code></td><td>set the active cluster version in the format '<major>.<minor>'</td></tr>
<tr><td><code>version</code></td><td>version</td><td><code>21.1-150</code></td><td>set the active cluster version in the format '<major>.<minor>'</td></tr>
</tbody>
</table>
3 changes: 3 additions & 0 deletions pkg/ccl/backupccl/system_schema.go
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,9 @@ var systemTableBackupConfiguration = map[string]systemBackupConfiguration{
systemschema.SQLInstancesTable.GetName(): {
shouldIncludeInClusterBackup: optOutOfClusterBackup,
},
systemschema.SpanConfigurationsTable.GetName(): {
shouldIncludeInClusterBackup: optOutOfClusterBackup,
},
}

// GetSystemTablesToIncludeInClusterBackup returns a set of system table names that
Expand Down
2 changes: 1 addition & 1 deletion pkg/cli/testdata/doctor/test_examine_cluster
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
debug doctor examine cluster
----
debug doctor examine cluster
Examining 40 descriptors and 41 namespace entries...
Examining 41 descriptors and 42 namespace entries...
ParentID 50, ParentSchemaID 29: relation "foo" (53): expected matching namespace entry, found none
Examining 4 jobs...
ERROR: validation failed
6 changes: 4 additions & 2 deletions pkg/cli/testdata/zip/partial1
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ debug zip --concurrency=1 --cpu-profile-duration=0s /dev/null
[node 1] 1 log file ...
[node 1] [log file ...
[node 1] requesting ranges... received response... done
[node 1] 42 ranges found
[node 1] 43 ranges found
[node 1] writing range 1... converting to JSON... writing binary output: debug/nodes/1/ranges/1.json... done
[node 1] writing range 2... converting to JSON... writing binary output: debug/nodes/1/ranges/2.json... done
[node 1] writing range 3... converting to JSON... writing binary output: debug/nodes/1/ranges/3.json... done
Expand Down Expand Up @@ -114,6 +114,7 @@ debug zip --concurrency=1 --cpu-profile-duration=0s /dev/null
[node 1] writing range 40... converting to JSON... writing binary output: debug/nodes/1/ranges/40.json... done
[node 1] writing range 41... converting to JSON... writing binary output: debug/nodes/1/ranges/41.json... done
[node 1] writing range 42... converting to JSON... writing binary output: debug/nodes/1/ranges/42.json... done
[node 1] writing range 43... converting to JSON... writing binary output: debug/nodes/1/ranges/43.json... done
[node 2] node status... converting to JSON... writing binary output: debug/nodes/2/status.json... done
[node 2] using SQL connection URL: postgresql://...
[node 2] retrieving SQL data for crdb_internal.feature_usage... writing output: debug/nodes/2/crdb_internal.feature_usage.txt...
Expand Down Expand Up @@ -236,7 +237,7 @@ debug zip --concurrency=1 --cpu-profile-duration=0s /dev/null
[node 3] 1 log file ...
[node 3] [log file ...
[node 3] requesting ranges... received response... done
[node 3] 42 ranges found
[node 3] 43 ranges found
[node 3] writing range 1... converting to JSON... writing binary output: debug/nodes/3/ranges/1.json... done
[node 3] writing range 2... converting to JSON... writing binary output: debug/nodes/3/ranges/2.json... done
[node 3] writing range 3... converting to JSON... writing binary output: debug/nodes/3/ranges/3.json... done
Expand Down Expand Up @@ -279,5 +280,6 @@ debug zip --concurrency=1 --cpu-profile-duration=0s /dev/null
[node 3] writing range 40... converting to JSON... writing binary output: debug/nodes/3/ranges/40.json... done
[node 3] writing range 41... converting to JSON... writing binary output: debug/nodes/3/ranges/41.json... done
[node 3] writing range 42... converting to JSON... writing binary output: debug/nodes/3/ranges/42.json... done
[node 3] writing range 43... converting to JSON... writing binary output: debug/nodes/3/ranges/43.json... done
[cluster] pprof summary script... writing binary output: debug/pprof-summary.sh... done
[cluster] hot range summary script... writing binary output: debug/hot-ranges.sh... done
6 changes: 4 additions & 2 deletions pkg/cli/testdata/zip/partial1_excluded
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ debug zip /dev/null --concurrency=1 --exclude-nodes=2 --cpu-profile-duration=0
[node 1] 1 log file ...
[node 1] [log file ...
[node 1] requesting ranges... received response... done
[node 1] 42 ranges found
[node 1] 43 ranges found
[node 1] writing range 1... converting to JSON... writing binary output: debug/nodes/1/ranges/1.json... done
[node 1] writing range 2... converting to JSON... writing binary output: debug/nodes/1/ranges/2.json... done
[node 1] writing range 3... converting to JSON... writing binary output: debug/nodes/1/ranges/3.json... done
Expand Down Expand Up @@ -114,6 +114,7 @@ debug zip /dev/null --concurrency=1 --exclude-nodes=2 --cpu-profile-duration=0
[node 1] writing range 40... converting to JSON... writing binary output: debug/nodes/1/ranges/40.json... done
[node 1] writing range 41... converting to JSON... writing binary output: debug/nodes/1/ranges/41.json... done
[node 1] writing range 42... converting to JSON... writing binary output: debug/nodes/1/ranges/42.json... done
[node 1] writing range 43... converting to JSON... writing binary output: debug/nodes/1/ranges/43.json... done
[node 2] skipping node... writing binary output: debug/nodes/2.skipped... done
[node 3] node status... converting to JSON... writing binary output: debug/nodes/3/status.json... done
[node 3] using SQL connection URL: postgresql://...
Expand Down Expand Up @@ -151,7 +152,7 @@ debug zip /dev/null --concurrency=1 --exclude-nodes=2 --cpu-profile-duration=0
[node 3] 1 log file ...
[node 3] [log file ...
[node 3] requesting ranges... received response... done
[node 3] 42 ranges found
[node 3] 43 ranges found
[node 3] writing range 1... converting to JSON... writing binary output: debug/nodes/3/ranges/1.json... done
[node 3] writing range 2... converting to JSON... writing binary output: debug/nodes/3/ranges/2.json... done
[node 3] writing range 3... converting to JSON... writing binary output: debug/nodes/3/ranges/3.json... done
Expand Down Expand Up @@ -194,5 +195,6 @@ debug zip /dev/null --concurrency=1 --exclude-nodes=2 --cpu-profile-duration=0
[node 3] writing range 40... converting to JSON... writing binary output: debug/nodes/3/ranges/40.json... done
[node 3] writing range 41... converting to JSON... writing binary output: debug/nodes/3/ranges/41.json... done
[node 3] writing range 42... converting to JSON... writing binary output: debug/nodes/3/ranges/42.json... done
[node 3] writing range 43... converting to JSON... writing binary output: debug/nodes/3/ranges/43.json... done
[cluster] pprof summary script... writing binary output: debug/pprof-summary.sh... done
[cluster] hot range summary script... writing binary output: debug/hot-ranges.sh... done
6 changes: 4 additions & 2 deletions pkg/cli/testdata/zip/partial2
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ debug zip --concurrency=1 --cpu-profile-duration=0 /dev/null
[node 1] 1 log file ...
[node 1] [log file ...
[node 1] requesting ranges... received response... done
[node 1] 42 ranges found
[node 1] 43 ranges found
[node 1] writing range 1... converting to JSON... writing binary output: debug/nodes/1/ranges/1.json... done
[node 1] writing range 2... converting to JSON... writing binary output: debug/nodes/1/ranges/2.json... done
[node 1] writing range 3... converting to JSON... writing binary output: debug/nodes/1/ranges/3.json... done
Expand Down Expand Up @@ -114,6 +114,7 @@ debug zip --concurrency=1 --cpu-profile-duration=0 /dev/null
[node 1] writing range 40... converting to JSON... writing binary output: debug/nodes/1/ranges/40.json... done
[node 1] writing range 41... converting to JSON... writing binary output: debug/nodes/1/ranges/41.json... done
[node 1] writing range 42... converting to JSON... writing binary output: debug/nodes/1/ranges/42.json... done
[node 1] writing range 43... converting to JSON... writing binary output: debug/nodes/1/ranges/43.json... done
[node 3] node status... converting to JSON... writing binary output: debug/nodes/3/status.json... done
[node 3] using SQL connection URL: postgresql://...
[node 3] retrieving SQL data for crdb_internal.feature_usage... writing output: debug/nodes/3/crdb_internal.feature_usage.txt... done
Expand Down Expand Up @@ -150,7 +151,7 @@ debug zip --concurrency=1 --cpu-profile-duration=0 /dev/null
[node 3] 1 log file ...
[node 3] [log file ...
[node 3] requesting ranges... received response... done
[node 3] 42 ranges found
[node 3] 43 ranges found
[node 3] writing range 1... converting to JSON... writing binary output: debug/nodes/3/ranges/1.json... done
[node 3] writing range 2... converting to JSON... writing binary output: debug/nodes/3/ranges/2.json... done
[node 3] writing range 3... converting to JSON... writing binary output: debug/nodes/3/ranges/3.json... done
Expand Down Expand Up @@ -193,5 +194,6 @@ debug zip --concurrency=1 --cpu-profile-duration=0 /dev/null
[node 3] writing range 40... converting to JSON... writing binary output: debug/nodes/3/ranges/40.json... done
[node 3] writing range 41... converting to JSON... writing binary output: debug/nodes/3/ranges/41.json... done
[node 3] writing range 42... converting to JSON... writing binary output: debug/nodes/3/ranges/42.json... done
[node 3] writing range 43... converting to JSON... writing binary output: debug/nodes/3/ranges/43.json... done
[cluster] pprof summary script... writing binary output: debug/pprof-summary.sh... done
[cluster] hot range summary script... writing binary output: debug/hot-ranges.sh... done
3 changes: 2 additions & 1 deletion pkg/cli/testdata/zip/testzip
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ debug zip --concurrency=1 --cpu-profile-duration=1s /dev/null
[node 1] requesting log file ...
[node 1] 0 log file ...
[node 1] requesting ranges... received response... done
[node 1] 42 ranges found
[node 1] 43 ranges found
[node 1] writing range 1... converting to JSON... writing binary output: debug/nodes/1/ranges/1.json... done
[node 1] writing range 2... converting to JSON... writing binary output: debug/nodes/1/ranges/2.json... done
[node 1] writing range 3... converting to JSON... writing binary output: debug/nodes/1/ranges/3.json... done
Expand Down Expand Up @@ -114,5 +114,6 @@ debug zip --concurrency=1 --cpu-profile-duration=1s /dev/null
[node 1] writing range 40... converting to JSON... writing binary output: debug/nodes/1/ranges/40.json... done
[node 1] writing range 41... converting to JSON... writing binary output: debug/nodes/1/ranges/41.json... done
[node 1] writing range 42... converting to JSON... writing binary output: debug/nodes/1/ranges/42.json... done
[node 1] writing range 43... converting to JSON... writing binary output: debug/nodes/1/ranges/43.json... done
[cluster] pprof summary script... writing binary output: debug/pprof-summary.sh... done
[cluster] hot range summary script... writing binary output: debug/hot-ranges.sh... done
18 changes: 15 additions & 3 deletions pkg/cli/testdata/zip/testzip_concurrent
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ zip
[cluster] using SQL address: ...
[cluster] using SQL address: ...
[node 1] 1 log file ...
[node 1] 42 ranges found
[node 1] 43 ranges found
[node 1] [log file ...
[node 1] [log file ...
[node 1] [log file ...
Expand Down Expand Up @@ -369,6 +369,10 @@ zip
[node 1] writing range 42: converting to JSON...
[node 1] writing range 42: done
[node 1] writing range 42: writing binary output: debug/nodes/1/ranges/42.json...
[node 1] writing range 43...
[node 1] writing range 43: converting to JSON...
[node 1] writing range 43: done
[node 1] writing range 43: writing binary output: debug/nodes/1/ranges/43.json...
[node 1] writing range 4: converting to JSON...
[node 1] writing range 4: done
[node 1] writing range 4: writing binary output: debug/nodes/1/ranges/4.json...
Expand All @@ -393,7 +397,7 @@ zip
[node 1] writing range 9: done
[node 1] writing range 9: writing binary output: debug/nodes/1/ranges/9.json...
[node 2] 1 log file ...
[node 2] 42 ranges found
[node 2] 43 ranges found
[node 2] [log file ...
[node 2] [log file ...
[node 2] [log file ...
Expand Down Expand Up @@ -644,6 +648,10 @@ zip
[node 2] writing range 42: converting to JSON...
[node 2] writing range 42: done
[node 2] writing range 42: writing binary output: debug/nodes/2/ranges/42.json...
[node 2] writing range 43...
[node 2] writing range 43: converting to JSON...
[node 2] writing range 43: done
[node 2] writing range 43: writing binary output: debug/nodes/2/ranges/43.json...
[node 2] writing range 4: converting to JSON...
[node 2] writing range 4: done
[node 2] writing range 4: writing binary output: debug/nodes/2/ranges/4.json...
Expand All @@ -668,7 +676,7 @@ zip
[node 2] writing range 9: done
[node 2] writing range 9: writing binary output: debug/nodes/2/ranges/9.json...
[node 3] 1 log file ...
[node 3] 42 ranges found
[node 3] 43 ranges found
[node 3] [log file ...
[node 3] [log file ...
[node 3] [log file ...
Expand Down Expand Up @@ -919,6 +927,10 @@ zip
[node 3] writing range 42: converting to JSON...
[node 3] writing range 42: done
[node 3] writing range 42: writing binary output: debug/nodes/3/ranges/42.json...
[node 3] writing range 43...
[node 3] writing range 43: converting to JSON...
[node 3] writing range 43: done
[node 3] writing range 43: writing binary output: debug/nodes/3/ranges/43.json...
[node 3] writing range 4: converting to JSON...
[node 3] writing range 4: done
[node 3] writing range 4: writing binary output: debug/nodes/3/ranges/4.json...
Expand Down
9 changes: 9 additions & 0 deletions pkg/clusterversion/cockroach_versions.go
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,10 @@ const (
// DatabasePlacementPolicy setting PLACEMENT for databases is supported in this
// version.
DatabasePlacementPolicy
// SpanConfigurationsTable adds the span configurations system table, to
// store all KV span configs.
SpanConfigurationsTable

// Step (1): Add new versions here.
)

Expand Down Expand Up @@ -519,6 +523,11 @@ var versionsSingleton = keyedVersions{
Key: DatabasePlacementPolicy,
Version: roachpb.Version{Major: 21, Minor: 1, Internal: 148},
},
{
Key: SpanConfigurationsTable,
Version: roachpb.Version{Major: 21, Minor: 1, Internal: 150},
},

// Step (2): Add new versions here.
}

Expand Down
5 changes: 3 additions & 2 deletions pkg/clusterversion/key_string.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pkg/keys/constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,7 @@ const (
DatabaseRoleSettingsTableID = 44
TenantUsageTableID = 45
SQLInstancesTableID = 46
SpanConfigurationsTableID = 47

// CommentType is type for system.comments
DatabaseCommentType = 0
Expand Down
1 change: 1 addition & 0 deletions pkg/migration/migrations/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ go_library(
"records_based_registry.go",
"retry_jobs_with_exponential_backoff.go",
"separated_intents.go",
"span_configurations.go",
"sql_instances.go",
"sql_stats.go",
"tenant_usage.go",
Expand Down
6 changes: 6 additions & 0 deletions pkg/migration/migrations/migrations.go
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,12 @@ var migrations = []migration.Migration{
NoPrecondition,
zonesTableForSecondaryTenants,
),
migration.NewTenantMigration(
"add the system.span_configurations table",
toCV(clusterversion.SpanConfigurationsTable),
NoPrecondition,
spanConfigurationsTableMigration,
),
}

func init() {
Expand Down
32 changes: 32 additions & 0 deletions pkg/migration/migrations/span_configurations.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
// Copyright 2021 The Cockroach Authors.
//
// Use of this software is governed by the Business Source License
// included in the file licenses/BSL.txt.
//
// As of the Change Date specified in that file, in accordance with
// the Business Source License, use of this software will be governed
// by the Apache License, Version 2.0, included in the file
// licenses/APL.txt.

package migrations

import (
"context"

"github.com/cockroachdb/cockroach/pkg/clusterversion"
"github.com/cockroachdb/cockroach/pkg/migration"
"github.com/cockroachdb/cockroach/pkg/sql/catalog/systemschema"
"github.com/cockroachdb/cockroach/pkg/startupmigrations"
)

func spanConfigurationsTableMigration(
ctx context.Context, _ clusterversion.ClusterVersion, d migration.TenantDeps,
) error {
if !d.Codec.ForSystemTenant() {
return nil
}

return startupmigrations.CreateSystemTable(
ctx, d.DB, d.Codec, d.Settings, systemschema.SpanConfigurationsTable,
)
}
1 change: 1 addition & 0 deletions pkg/sql/catalog/bootstrap/metadata.go
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,7 @@ func addSystemDescriptorsToSchema(target *MetadataSchema) {
target.AddDescriptor(systemschema.DatabaseRoleSettingsTable)
target.AddDescriptorForSystemTenant(systemschema.TenantUsageTable)
target.AddDescriptor(systemschema.SQLInstancesTable)
target.AddDescriptorForSystemTenant(systemschema.SpanConfigurationsTable)

// Adding a new system table? It should be added here to the metadata schema,
// and also created as a migration for older clusters. The includedInBootstrap
Expand Down
1 change: 1 addition & 0 deletions pkg/sql/catalog/catconstants/constants.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ const (
DatabaseRoleSettingsTableName SystemTableName = "database_role_settings"
TenantUsageTableName SystemTableName = "tenant_usage"
SQLInstancesTableName SystemTableName = "sql_instances"
SpanConfigurationsTableName SystemTableName = "span_configurations"
)

// Oid for virtual database and table.
Expand Down
Loading

0 comments on commit 4054d34

Please sign in to comment.