Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider strategy for testing upgrades across multiple versions #27

Open
EvanBoyle opened this issue Oct 31, 2023 · 2 comments
Open

Consider strategy for testing upgrades across multiple versions #27

EvanBoyle opened this issue Oct 31, 2023 · 2 comments
Labels
kind/engineering Work that is not visible to an external user

Comments

@EvanBoyle
Copy link

We recently added tests for GCP v6 -> v7. We know that a significant number of our customers don't upgrade in a timely manner, so we should consider how we continue to accrue testing across versions in future upgrades. For instance, can we have tests for both v7->v8 and v6->v8?

@EvanBoyle EvanBoyle added kind/engineering Work that is not visible to an external user needs-triage Needs attention from the triage team labels Oct 31, 2023
@t0yv0
Copy link
Member

t0yv0 commented Oct 31, 2023

Thank you for filing this. Without making any changes to v0.0.2 of this library, one can add several distinct tests with a different baseline version, and that should work in principle to test v7->latest, v6->latest which would work if latest=v8.

That said it seems that it should be easy to support testing arbitrary X->Y upgrades and having a documented way to do that, so if one wanted to test v6->v7, v7->v8 or v7->v8 that should be doable. We might need to make some small tweaks to enable this.

@t0yv0
Copy link
Member

t0yv0 commented Nov 15, 2023

@guineveresaenger brings up the need to figure out automation to support this feature as well. Upon a new release of a provider, we need to add (or move) a new baseline, and to support test accelerators we need to perform recording of the provider's behavior at the new baseline which can be time-consuming. Ideally this chore is automated and happens in a way that is non-obtrusive for developers.

@mjeffryes mjeffryes removed the needs-triage Needs attention from the triage team label Feb 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/engineering Work that is not visible to an external user
Projects
None yet
Development

No branches or pull requests

3 participants