pkg/util/debugutil: introduce debug.Stack() wrapper

[arjunmahishi]

Until now, all stack traces were treated as untyped strings and hence redacted. Stack traces don't need to be redacted. Because they don't contain any PII. This commit fixes that with the following changes:

• Introduce a new function Stack() in the debugutil package as a replacement for the standard debug.Stack(). This new function wraps the stack trace contents in redact.Safe() to make sure that it does not get redacted.

• Replace the all the current uses of debug.Stack() across the codebase with debugutil.Stack()

• Add a linter rule to block future uses of debug.Stack(). The linter error will nudge users to use debugutil.Stack() instead.

Part of: CRDB-15292
Epic: CRDB-37533
Release note: None

[cockroach-teamcity]

This change is 

[dhartunian]

Can you split the 3 bullet points into separate commits? Easier to review search/replace operations when they're confined to a commit.

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @aa-joshi, @arjunmahishi, @itsbilal, @kyle-a-wong, @nameisbhaskar, and @vidit-bhat)

---

pkg/util/debugutil/debugutil.go line 53 at r1 (raw file):

// unnecessary redaction.
func Stack() interfaces.SafeValue {
	return redact.Safe(string(debug.Stack()))

I believe string will allocate a copy of the stack byte slice.
Can you do this via type alias and avoid an allocation?

type SafeStack []byte

func (SafeStack) SafeValue() {}

SafeStack(debug.Stack())

Alternatively, you can use unsafe.String if the above doesn't work.

[itsbilal]

mostly a transit-by review as I got tagged

[itsbilal]

Given we've had issues in the past with high system CPU usage due to excessive use of debug.Stack in error handling pathways, we should probably add a warning comment here to the effect of "Calling debug.Stack grabs system-level locks and could cause high system CPU usage resulting in the Go runtime to lock up if called too frequently, even if called only in error-handling pathways. Use sporadically and only when necessary."