roachprod: add COCKROACH_UPGRADE_TO_DEV_VERSION to DefaultEnvVars

Recent changes to cockroach_versions (logic) now require setting
COCKROACH_UPGRADE_TO_DEV_VERSION environment variable in order to
allow upgrading a stable release data-dir to a dev. version.
The PR [1] which introduced this env. var. did not correctly backport
the change to all the roachtests which perform this type of upgrade.

Since all roachtests which exercise upgrade paths are intended to
test this upgrade scenario, we set COCKROACH_UPGRADE_TO_DEV_VERSION
by default in roachprod instead of polluting the roachtests with more
config. settings.

Consequently, MakeClusterSettings now returns the default
ClusterSettings which includes COCKROACH_UPGRADE_TO_DEV_VERSION;
generateStartCmd ensures it's passed into cockroach env. via start.sh.

Release note: None

Release justification: bug fix in roachtests.

Resolves:
  cockroachdb#87675
  cockroachdb#87687

[1] cockroachdb#87468

pkg/cmd/roachtest/tests/versionupgrade.go

@@ -431,7 +431,6 @@ func upgradeNodes(
 
 		binary := uploadVersion(ctx, t, c, c.Node(node), newVersion)
 		settings := install.MakeClusterSettings(install.BinaryOption(binary))
-		settings.Env = append(settings.Env, "COCKROACH_UPGRADE_TO_DEV_VERSION=true")
 		c.Start(ctx, t.L(), startOpts, settings, c.Node(node))
 
 		// We have seen cases where a transient error could occur when this

pkg/roachprod/config/config.go

@@ -88,6 +88,11 @@ func DefaultEnvVars() []string {
 		"COCKROACH_ENABLE_RPC_COMPRESSION=false",
 		// Get rid of an annoying popup in the UI.
 		"COCKROACH_UI_RELEASE_NOTES_SIGNUP_DISMISSED=true",
+		// Allow upgrading a stable release data-dir to a dev version.
+		// N.B. many roachtests which perform upgrade scenarios require this env. var after changes in [1]; otherwise,
+		// the tests will fail even on release branches when attempting to upgrade previous (stable) release to an alpha.
+		// [1] https://github.com/cockroachdb/cockroach/pull/87468
+		"COCKROACH_UPGRADE_TO_DEV_VERSION=true",
 	}
 }