catalog: enforce uniqueness of id column in system.namespace

[postamar]

Such a validation check would prevent nasty bugs from happening. Now that draining names are no longer a thing, nothing prevents us from doing this. We can't do this by adding a unique constraint because we happily write into that table directly using KV-level Puts so we're going to have to be a bit more clever about it.

Jira issue: CRDB-17737

[ajwerner]

This is more or less what I think we intended to do with the namespace validation, but in practice that doesn't achieve the intended goal.

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 84 to 88 in 2623bae

	descGetterErr = vdg.addNamespaceEntries(ctx, descriptors, vd)
	if descGetterErr != nil {
	vea.reportDescGetterError(collectingNamespaceEntries, descGetterErr)
	return vea.errors
	}

[postamar]

Now that draining names have been removed, we should also be more strict with what is in system.namespace but shouldn't be. This might have helped with https://cockroachlabs.atlassian.net/browse/SREOPS-4907

[postamar]

We want the descs.Collection:

1. to do CPuts when writing to the descriptors table to prevent clobbering of that table.
2. upon doing a Put to the namespace table, check that the referenced descriptor back-references the namespace key we're about to insert.

Presumably, this descriptor will be held in memory by the Collection at that point already.

[postamar]

@Xiang-Gu expressed an interest in taking this on.

[Xiang-Gu]

One milestone of

to do CPuts when writing to the descriptors table to prevent clobbering of that table.

will be achieved when PR #88844 is merged. This change will tighten the logic whenever we write to the sytem.descriptor table, and should be able to guard against a wide range of descriptor table corruption, including secondary corruptions like this issue.

For now, I'll step away and focus my time on some other development work for the next release.