add Kubernetes End to End tests to the ISCSI Driver

[mkimuram]

What type of PR is this?
/kind feature

What this PR does / why we need it:
This PR adds End to End tests to the ISCSI plugin.

Which issue(s) this PR fixes:

Special notes for your reviewer:

• This PR updates the driver to Kubernetes-1.15
• This PR leverages the same way as nfs plugin does in add Kubernetes End to End tests to the NFS Driver csi-driver-nfs#10

Does this PR introduce a user-facing change?:

NONE

[mkimuram]

@mathu97 @j-griffith @wongma7 @msau42 Please take a look.

[k8s-ci-robot]

@j-griffith: GitHub didn't allow me to assign the following users: me.

Note that only kubernetes-csi members and repo collaborators can be assigned and that issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign me

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[j-griffith]

/assign j-griffith

[mkimuram]

Below error seems to be caused by go version. We need to update go version of the build system to 1.12 or resolve dependency not to update to k8s 1.15.

### test-go:
go test `go list ./... | grep -v 'vendor' ` 
?   	github.com/kubernetes-csi/csi-driver-iscsi/cmd/iscsiplugin	[no test files]
# github.com/kubernetes-csi/csi-driver-iscsi/vendor/k8s.io/component-base/cli/flag
vendor/k8s.io/component-base/cli/flag/ciphersuites_flag.go:80:18: undefined: tls.VersionTLS13
make: *** [test-go] Error 2

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mkimuram
To complete the pull request process, please assign j-griffith
You can assign the PR to them by writing /assign @j-griffith in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mkimuram]

Build issue is now resolved by changing the dependency back to k8s 1.14.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[msau42]

@jsafrane did your change to add iscsi e2e jobs address this?

[jsafrane]

This iSCSI job tests in-tree iSCSI volume plugin: https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes/sig-storage/sig-storage-gce-config.yaml#L154

It can be taken as inspiration how to run e2e tests on Ubuntu (it has iscsid installed on the host). Otherwise you get the default GCI image on the host and you need to run iscsid inside the driver pod. Which may be ok, because there will be only one such pod on the host.

[mkimuram]

@jsafrane

Thank you for sharing how you enabled in-tree iSCSI e2e test.
I've checked the codes and configurations for k8s CI jobs from the link (I haven't yet understand them completely, though.)

To make e2e test CI available for CSI iSCSI driver, I think that we need below five steps:

1. Create e2e test for CSI iSCSI driver (This PR)
   (CSI iSCSI driver needs to create custom e2e binary, because dynamic provisioning is not implemented. https://github.com/kubernetes-csi/docs/blob/master/book/src/functional-testing.md#setting-up-end-to-end-tests-for-your-csi-plugin )

2. Update prow.sh in kubernetes-csi/csi-release-tools repo to run custom e2e binary
   (Current prow.sh only seems to run k8s e2e test, so we need to make custom e2e binary run. https://github.com/kubernetes-csi/csi-release-tools/blob/master/prow.sh#L781 )

3. Make kubernetes-csi/csi-release-tools sync to kubernetes-csi/csi-driver-iscsi
   https://github.com/kubernetes-csi/csi-release-tools#sharing-and-updating

4. Create .prow.sh in kubernetes-csi/csi-driver-iscsi

5. Update config/jobs/kubernetes-csi/gen-jobs.sh in kubernetes/test-infra repo and generate config for making kubernetes-csi/csi-driver-iscsi repo tested
   https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes-csi/gen-jobs.sh#L66

And for step 5, using ubuntu image will be the key for CSI iSCSI driver, too.

[msau42]

@mkimuram this sounds like something that maybe we can do entirely in kubernetes-csi/csi-driver-iscsi? In that case, we shouldn't need any changes in k/k

[mkimuram]

@msau42

I've just started to look through the codes around CI, so I'm not 100% sure.
However, csi-driver-host-path seems to have its own CI jobs for e2e under kubernetes-csi repo, so other csi drivers will also be able to achieve the similar goal, if there is no big blocker.

(And for me, the big blockers were lack of iscsid on the host and lack of my knowledge on k8s CI jobs, before I saw Jan's config above.)

[msau42]

Yes, it is totally possible to run the K8s e2e test suites on repos outside k/k thanks to your work! And we do it for kubernetes-csi/csi-driver-host-path, and many other csi drivers too. I am fine with setting up e2e jobs on the csi-driver-iscsi repo too if you're interested, but I do think the requirement of iscsid on the host may make things challenging. The kubernetes-csi jobs use kind for their clusters, where nodes are containers, and their nodes have very minimal packages available. We could potentially look into building custom node images (still unsure how technically feasible it is), or just have a completely different infra that brings up gce ubuntu nodes.

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closed this PR.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.