Also handle BWC tests as part of version increment automation

[ankitkala]

Is your feature request related to a problem? Please describe

There is an ongoing effort for automating the process to increment the version for the whole project(link). As part of the effort, we should also be handling version bump for BWC tests.

For BWC tests, we need to keep track of older version that you want to test against. Internally BWC test will spinup the cluster in older version and then test various upgrade scenarios(rolling upgrade, mixed cluster state, etc). When we're bumping up the major version for each plugin, we also need to bump up the older version for BWC tests.
Here is the example for CCR plugin but the same problem applies to all the plugins.

Describe the solution you'd like

Option 1: Version increment automation should also handle bumping up the older version for BWC tests.
Option 2: Add a checklist item for every major release to also bump up the version for BWC tests,

Describe alternatives you've considered

No response

Additional context

No response

[prudhvigodithi]

[Triage] Hey @ankitkala the version increment process is generic and is not related to any BWC tests, we recommend this could be handled by plugin team per release branch, since each plugin handle the BWC tests differently, its not feasible to add it to the version increment task updateVersion.

[ankitkala]

Hey @prudhvigodithi,
I understand that that each plugin handles BWC tests differently. But atleast we can then go with 2nd option where we have this as one of the release checklist item on every major version bump.
cc: @dblock

[prudhvigodithi]

Hey @ankitkala Since we are proposing to follow Core branching strategy, for BWC tests plugin teams can handle this during repo branching, example when 2.x is created, we can have BWC version change handle here, so this will point to latest 1.x version, once we have branch for 3.x, this will have latest 2.x version for BWC, since the minor version branches (2.2 or 3.1) are cut from 2.x and 3.x, they should by default have the right version for BWC. Assuming BWC tests done for the major version and not for minor or patch.

[prudhvigodithi]

Hey @ankitkala closing this issue as we can track the progress on #3063, please feel free to re-open if required.