release-22.2: sql,opt: improve SHOW TABLES and virtual tables with many databases

[ajwerner]

Backport 6/6 commits from #90116.

/cc @cockroachdb/release

---

information_schema and pg_catalog are always scoped to a database. Before this change, we would always look up all descriptors to materialize virtual tables, even if we only care about descriptors in the current database. This PR adds a new cluster setting (so that backports can be made kosher) to enable a new approach: scan the namespace table to find the relevant descriptors and then only resolve those.

On my local machine where I loaded up a few megabytes of descriptors into 6k databases, a simple query like SELECT * FROM information_schema.sequences takes 1.2s without the setting enabled and 1ms with it.

Release justification: Fixes an important issue for using many databases.

Fixes #90115

Epic: CRDB-20865

Release note (performance improvement): Tables in pg_catalog and
information_schema (when not explicitly referenced as
"".information_schema) may now be much faster if the current database has
a small number of relations relative to the total number in the cluster.

[cockroach-teamcity]

This change is 

[rytaft]

First commit

Reviewed 3 of 4 files at r1, 1 of 1 files at r2.
Reviewable status: complete! 1 of 0 LGTMs obtained

[ajwerner]

@rafiss can I get a look at this and can you help me decide whether I should default this to off on release-22.2?

[rafiss]

this lgtm, with some minor nits that maybe should be addressed in master as well. as far as the default setting value, i feel fine about leaving it as true. one thing i wonder though is how do we envision that someone would know to disable it if they need to? it doesn't seem like something we want to spend time training TSE on

Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @ajwerner and @rytaft)

---

pkg/sql/crdb_internal.go line 488 at r4 (raw file):

	indexes: []virtualIndex{
		{
			populate: func(ctx context.Context, unwrappedConstraint tree.Datum, p *planner, _ catalog.DatabaseDescriptor, addRow func(...tree.Datum) error) (matched bool, err error) {

nit: add a comment to each populate to make it clear which index is which

---

pkg/sql/resolver.go line 988 at r3 (raw file):

}

var metamorphicDeadultUseIndexLookupForDescriptorsInDatabase = util.ConstantWithMetamorphicTestBool(

nit: typo

---

pkg/sql/delegate/show_tables.go line 113 at r4 (raw file):

		estimatedRowCount,
		estimatedRowCountJoin,
		tree.NewDString(name.CatalogName.String()),

nit: did you test this query with a database named like "Mixed-Case"? i believe CatalogName.String() will add those quotes, but for this query we don't want that, so maybe it should just be string(name.CatalogName). also, the NewDString is not needed.

[ajwerner]

@rafiss TFTR! I added the commits from #91292 to this PR to address your feedback.