Add a test OLM upgrade path workflow

[maleck13]

What

In the Kuadrant operator github actions add a workflow that allows specifying two existing versions (upgrade from and upgrade to) and execute an upgrade from these versions using OLM.

Ideally this workflow would simply call a script or make command that could also be ran locally or in another CI flow against any target cluster that has OLM installed on it. For this workflow it should be fine to test on a kind cluster that has OLM installed on it.

[Boomatang]

For RHOAM I have done a some work to be able to do this locally. You could give it a version "A" as a start point and version "X" as a end point. It would work out the chain based on the release versions. There maybe some work there that we can reuse.
https://github.com/Boomatang/orc-dev-cli/blob/main/docs/commnads/rhoam.md#index

[maleck13]

Interesting. I was also looking with @eguzki at the upgrade command from operator-sdk. We could possibly avoid having to mess with Catalogs https://sdk.operatorframework.io/docs/cli/operator-sdk_run_bundle-upgrade/

[Boomatang]

One of the reason, I went at the time with creating standalone catalogs was for local testing. I felt that you would want to be able to quickly switch back to past versions. I am not sure if the upgrade command in the operator-sdk would allow that. It would also be the case if there is a pipeline that does upgrade testing and is failing, we may need to be able to install the previous state before the upgrade was tried. Operator-sdk can simplify this all the better.

One thing that should be considered is be able do the upgrade from n-2, or more. There was some issues that were notice when going from n-2 to n, but the issue didn't happen with n-1 to n. What ever system is used would need to be able to go from the oldest version we are willing to support to the latest.

[maleck13]

Might be possible to achieve this with a set of calls to the bundle upgrade command ? like install catalog v1.0 run upgrade to v1.0.1 then run upgrade to v1.02? Anyway lets discuss the options in further detail and decide on a direction forward