Prow testing #265
Prow testing #265
Conversation
In repos that have a test/e2e, that test suite should be run separately because it depends on a running cluster.
This is an unmodified copy of kubernetes/hack/verify-shellcheck.sh revision d5a3db003916b1d33b503ccd2e4897e094d8af77.
check vendor directory
In repos that have a test/e2e, that test suite should be run separately because it depends on a running cluster.
build.make: avoid unit-testing E2E test suite
These are the modifications that were necessary to call this outside of Kubernetes. The support for excluding files from checking gets removed to simplify the script. It shouldn't be needed, because linting can be enabled after fixing whatever scripts might fail the check.
By default this only tests the scripts inside the "release-tools" directory, which is useful when making experimental changes to them in a component that uses csi-release-tools. But a component can also enable checking for other directories.
This enables testing of other repos and of this repo itself inside
Prow. Currently supported is unit testing ("make test") and E2E
testing (either via a local test suite or the Kubernetes E2E test
suite applied to the hostpath driver example deployment).
The script passes shellcheck and uses Prow to verify that for future
PRs.
verify shellcheck
The travis.yml is now the only place where the Go version for the component itself needs to be configured.
Using the same (recent) Go version for all Kubernetes versions can break for older versions when there are incompatible changes in Go. To avoid that, we use exactly the minimum version of Go required for each Kubernetes version. This is based on the assumption that this combination was tested successfully. When building the E2E suite from Kubernetes (the default) we do the same, but still allow building it from elsewhere. We allow the Go version to be empty when it doesn't matter.
While switching back and forth between release-1.13 and release-1.14 locally, there was the problem that the local kind image kept using the wrong kubelet binary despite rebuilding from source. The problem went away after cleaning the Bazel cache. Exact root cause unknown, but perhaps using unique tags and properly cleaning the repo helps. If not, then the unique tags at least will show what the problem is.
Instead of always using the latest E2E tests for all Kubernetes versions, the plan now is to use the tests that match the Kubernetes version. However, for 1.13 we have to make an exception because the suite for that version did not support the necessary --storage.testdriver parameter.
The temporary fork was merged, we can use the upstream repo again.
Prow testing
This ensures that also new, currently unknown alpha gates are enabled when testing against a future Kubernetes versions. For all currently known Kubernetes versions we just use the minimal set of alpha gates, which ensures that we don't miss any of them in our documentation.
"grep -w" treated "serial-alpha" as two words and therefore CSI_PROW_TESTS sometimes ran too many tests.
The previous logic failed for canary jobs, those also deploy a recent driver. Instead of guessing what driver gets installed based on job parameters, check what really runs in the cluster and base the decision on that. We only need to maintain this blacklist for 1.0.x until we replace it with 1.1.0, then this entire hostpath_supports_block can be removed.
When running only some tests, sometimes extra, unnecessarily work was done, like bringing up the cluster without alpha gates.
Not all environments have Docker. The simplifying assumption here is that if the Docker command is available, it's also usable.
When KinD fails in a Prow job, we need additional information to understand why it failed.
It turned out to not work. Instead of reverting the commit which introduced this, let's better document this explicitly.
prow enhancements
This was already meant to be done earlier in cda2fc5. While at it, extend the permanent TODO with guidance on future feature gates.
prow.sh: remove AllAlpha=all, part II
k8s-ci-robot
added
approved
k8s-ci-robot
removed
the
approved
|
/test pull-kubernetes-csi-external-provisioner-1-13-on-kubernetes-1-13 |
build.make: Replace 'return' to 'exit' to fix shellcheck error
|
/test pull-kubernetes-csi-external-provisioner-1-13-on-kubernetes-master |
This pulls in kubernetes-csi/csi-driver-host-path#37, which turned out to be necessary for pre-submit testing of the livenessprobe.
prow.sh: update csi-driver-host-path
|
/test pull-kubernetes-csi-external-provisioner-1-13-on-kubernetes-master |
k8s-ci-robot
added
the
do-not-merge/work-in-progress
Whether a component supports sanity testing depends on the component. For example, csi-driver-host-path enables it because it makes sense there (and only there). Letting the prow.sh script decide whether it actually runs simplifies the job definitions in test-infra.
prow.sh: skip sanity testing if component doesn't support it
|
/test pull-kubernetes-csi-external-provisioner-1-13-on-kubernetes-1-13 |
k8s-ci-robot
removed
the
do-not-merge/work-in-progress
|
I'm surprised the tests are passing. Didn't we break backwards compatibility with the new RBAC rules? |
|
The periodic tests failed because the unmodified RBAC rules: The pull jobs intentionally use the modified RBAC rules and therefore tests pass. That is done this way to allow breaking changes which change code and RBAC rules. Otherwise it wouldn't be possible to merge such changes without fiddling with testing. Note that the initial PR changed the code but not the RBAC rules. That would have been caught by these pull jobs. |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: msau42, pohly The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
Commit summary: Squashed 'release-tools/' changes from 227577e00..734c2b950 [734c2b950](kubernetes-csi/csi-release-tools@734c2b950) Merge [pull request kubernetes-csi#265](kubernetes-csi/csi-release-tools#265) from Rakshith-R/consider-main-branch [f95c855be](kubernetes-csi/csi-release-tools@f95c855be) Merge [pull request kubernetes-csi#262](kubernetes-csi/csi-release-tools#262) from huww98/golang-toolchain [3c8d966fe](kubernetes-csi/csi-release-tools@3c8d966fe) Treat main branch as equivalent to master branch [e31de525b](kubernetes-csi/csi-release-tools@e31de525b) Merge [pull request kubernetes-csi#261](kubernetes-csi/csi-release-tools#261) from huww98/golang [fd153a9e2](kubernetes-csi/csi-release-tools@fd153a9e2) Bump golang to 1.23.1 [a8b3d0504](kubernetes-csi/csi-release-tools@a8b3d0504) pull-test.sh: fix "git subtree pull" errors [6b05f0fcc](kubernetes-csi/csi-release-tools@6b05f0fcc) use new GOTOOLCHAIN env to manage go version git-subtree-dir: release-tools git-subtree-split: 734c2b950c4b31f64b63052c64ffa5929d1c9b97 Signed-off-by: Mario Valderrama <[email protected]>
98f23071d Merge pull request kubernetes-csi#260 from TerryHowe/update-csi-driver-version e9d8712d0 Merge pull request kubernetes-csi#259 from stmcginnis/deprecated-kind-kube-root faf79ff66 Remove --kube-root deprecated kind argument 734c2b950 Merge pull request kubernetes-csi#265 from Rakshith-R/consider-main-branch f95c855be Merge pull request kubernetes-csi#262 from huww98/golang-toolchain 3c8d966fe Treat main branch as equivalent to master branch e31de525b Merge pull request kubernetes-csi#261 from huww98/golang fd153a9e2 Bump golang to 1.23.1 a8b3d0504 pull-test.sh: fix "git subtree pull" errors 6b05f0fcc use new GOTOOLCHAIN env to manage go version 18b6ac6d2 chore: update CSI driver version to 1.15 git-subtree-dir: release-tools git-subtree-split: 98f23071d946dd3de3188a7e1f84679067003162
No description provided.