Add upgrade test into E2E tests

[danfengliu]

Signed-off-by: danfengl [email protected]

Thank you for contributing to Velero!

Please add a summary of your change

Add upgrade test in E2E test.

Does your change fix a particular issue?

Fixes #4029

Please indicate you've done the following:

• Accepted the DCO. Commits without the DCO will delay acceptance.
• Created a changelog file or added /kind changelog-not-required.
• Updated the corresponding documentation in site/content/docs/main.

[zubron]

Thanks for this, @danfengliu! This is looking great so far :) I've done a lot of upgrade testing recently so I know this is a really complex test to write so thank you for taking this on :)

I have a few comments and suggestions. My main request is that we should be using the upgrade instructions from the website as I don't think calling install again will replace the CRDs and update the deployment. Also, I think that it should be the responsibility of the runner of these tests to provide the binary to use rather than this code downloading it.

I think the flow of the test should be

1. Install with the upgrade-from binary (and don't set the image to use, let the binary use the default image)
2. Perform the backup using upgrade-from binary
3. Upgrade by explicitly installing the new CRDs and upgrading the image in the deployment/daemonset using the CLI and image that are being tested (the PR/nightly build). In future we should consider adding an "upgrade" command to the CLI to improve this user experience, but that's a problem for another time :)
4. Use the new CLI that's being tested to perform the restore.

I hope my comments makes sense! Let me know if anything is unclear 👍

[ywk253100]

@danfengliu Please resolve the conflict either

[zubron]

Thanks for making the suggested changes :) I have a few more comments but it's looking good 👍

[ywk253100]

Thanks

[zubron]

Thanks for this, @danfengliu! I am happy for this to be merged, I just want to check if we can go back to using the upstream version of kibishii rather than a fork.

[zubron]

LGTM - thanks!