Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

e2e: testing reliability #7701

Merged
merged 3 commits into from
Apr 13, 2020
Merged

e2e: testing reliability #7701

merged 3 commits into from
Apr 13, 2020

Conversation

tgross
Copy link
Member

@tgross tgross commented Apr 13, 2020

This changeset makes 3 changes to improve reliability and usability of e2e tests:

  • pin the Docker image version of the CSI plugins, which recently caused nightly tests to fail because of a bad image push from upstream
  • ensure all the jobs get cleaned up during CSI's e2e testing, even the ones we expect to "never fail" (because, see above 😀)
  • update the Makefile for make dev-cluster to accept an env var override.

@tgross tgross added the theme/testing Test related issues label Apr 13, 2020
@tgross tgross added this to the 0.11.1 milestone Apr 13, 2020
@tgross tgross requested review from langmartin and notnoop April 13, 2020 14:09
langmartin
langmartin approved these changes Apr 13, 2020
View reviewed changes
e2e/terraform/Makefile
@@ -1,4 +1,4 @@
NOMAD_SHA := $(shell git rev-parse HEAD)
NOMAD_SHA ?= $(shell git rev-parse HEAD)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL - I've been using make FOO=BAR target so didn't notice this issue much before. Thanks!

notnoop
notnoop approved these changes Apr 13, 2020
View reviewed changes
Copy link
Contributor

@notnoop notnoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I'd be nice to be able to test against the latest versions too - at least to get an early warning of breakage; though this doesn't seem like the right place. Might make sense for periodic checks/review (e.g. along with golang versions, golang.org/x modules, etc).

e2e/terraform/Makefile
@@ -1,4 +1,4 @@
NOMAD_SHA := $(shell git rev-parse HEAD)
NOMAD_SHA ?= $(shell git rev-parse HEAD)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL - I've been using make FOO=BAR target so didn't notice this issue much before. Thanks!

@tgross tgross merged commit cafdcc9 into master Apr 13, 2020
@tgross tgross deleted the b-csi-e2e-no-efs-start branch April 13, 2020 14:25
@tgross
Copy link
Member Author

tgross commented Apr 13, 2020

I'd be nice to be able to test against the latest versions too - at least to get an early warning of breakage; though this doesn't seem like the right place.

It looks like for these two plugins that they do GitHub releases for each tag they push, including (very brief) RC window. So I've subscribed to those releases and that'll give us a notification to update these tests at least.

@tgross tgross mentioned this pull request May 12, 2020
Closed
@github-actions
Copy link

github-actions bot commented Jan 9, 2023

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 9, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@notnoop notnoop notnoop approved these changes

@langmartin langmartin langmartin approved these changes

Assignees
No one assigned
Labels
theme/testing Test related issues
Projects
None yet
Milestone
0.11.1
Development

Successfully merging this pull request may close these issues.
3 participants
@tgross @langmartin @notnoop