[Elastic Agent][Meta] Agent 'upgrade test' coverage needs

[EricDavisX]

Michal and I discussed and want to merge the upgrade-agent test PR as is with the following additional plans:
[added recently]

• we will re-enable the test ASAP on master, which was disabled here disable upgrade tests #537 - it was disabled because it is failing on PR branch tests, and preventing PR CI tests from passing. so we need to re-work the test or decide to give in and find a way to skip it for *just PR tests during CI (so it can run in the nightlys)

[existing]

• we will re-purpose the 'agent_stale' version to be a current release-1 version and run the same test
• the above may be done already from prior work... need to check.
• we will add a new 'prior_snapshot' version to be the same release line but 1 version older (or the same version if the prior doesn't exist for some reason)
  this second test will help validate both sides of the upgrade code, now on the current release line (instead of only testing it on the already released 'previous stale' version).
  ... instead of the prior snapshot, do we want to do a shorter-term 'upgrade with the same version' test, which at least covers the current code, even if the validation is weaker
• need to cover at least a basic case for Agent 'Rollback' - perhaps we can manually delete / change some filesystem items and then attempt the upgrade, knowing it will fail and can track certain logs to help verify it attempted it
• some requirement from Beats 27773 is missing in the test, I believe it has to do with setting additional Input options and ensuring that they are retained (and that no duplicate data is retrieved) - those are 2 new facets to the test we can improve on
• Add in a matrix / list of versions we could test upgrading from (to the latest) to see cross-version breaks that we know can occur
• other items to consider itemizing?

follow on to this pr: #506

[EricDavisX]

@mdelapenya when we get in on Monday... can you and me and Michal talk (in any grouping) about how we could re-enable the upgrade test for just the nightly runs.

[EricDavisX]

it is re-enabled now. there remains some extra feature work we wanted to do here that I will keep the ticket open for

[mdelapenya]

I think we can close this one, as the upgrade scenarios are in place