Add integration tests on Git action #69

jingxu97 · 2020-08-31T17:50:20Z

Add integration tests

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

k8s-ci-robot · 2020-08-31T17:50:22Z

@jingxu97: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ddebroy · 2020-09-01T19:47:47Z

/lgtm
/approve

k8s-ci-robot · 2020-09-01T19:47:51Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ddebroy, jingxu97

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [ddebroy,jingxu97]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

andyzhangx · 2020-09-08T03:54:52Z

can you continue working on the unit test failures @mayankshah1607 , thanks:

C:\Users\xiazhang\go\src\github.com\kubernetes-csi\csi-proxy>go test -v -race ./integrationtests/...
=== RUN   TestAPIGroups
=== RUN   TestAPIGroups/happy_path_with_v1alpha1
=== RUN   TestAPIGroups/overflow_with_v1alpha1
=== RUN   TestAPIGroups/happy_path_with_v1alpha2
=== RUN   TestAPIGroups/overflow_with_v1alpha2
=== RUN   TestAPIGroups/with_v1
--- FAIL: TestAPIGroups (0.01s)
    --- FAIL: TestAPIGroups/happy_path_with_v1alpha1 (0.00s)
        api_groups_test.go:35:
                Error Trace:    api_groups_test.go:35
                Error:          Not equal:
                                expected: 56
                                actual  : 0
                Test:           TestAPIGroups/happy_path_with_v1alpha1
    --- FAIL: TestAPIGroups/overflow_with_v1alpha1 (0.00s)
        api_groups_test.go:48:
                Error Trace:    api_groups_test.go:48
                Error:          Expected nil, but got: &v1alpha1.ComputeDoubleResponse{Response32:0, XXX_NoUnkeyedLiteral:struct {}{}, XXX_unrecognized:[]uint8(nil), XXX_sizecache:0}
                Test:           TestAPIGroups/overflow_with_v1alpha1
        api_groups_test.go:49:
                Error Trace:    api_groups_test.go:49
                Error:          Expected value not to be nil.
                Test:           TestAPIGroups/overflow_with_v1alpha1
    --- PASS: TestAPIGroups/happy_path_with_v1alpha2 (0.00s)
    --- PASS: TestAPIGroups/overflow_with_v1alpha2 (0.00s)
    --- PASS: TestAPIGroups/with_v1 (0.00s)
=== RUN   TestNewAPIGroup
--- FAIL: TestNewAPIGroup (16.72s)
    utils.go:117:
                Error Trace:    utils.go:117
                                                        csi_api_gen_test.go:107
                                                        csi_api_gen_test.go:25
                Error:          Expected nil, but got: &os.PathError{Op:"open", Path:"", Err:0x2}
                Test:           TestNewAPIGroup
                Messages:       unable to read ""
    csi_api_gen_test.go:98:
                Error Trace:    csi_api_gen_test.go:98
                                                        panic.go:563
                                                        testing.go:653
                                                        utils.go:117
                                                        csi_api_gen_test.go:107
                                                        csi_api_gen_test.go:25
                Error:
                Test:           TestNewAPIGroup
=== RUN   TestDiskAPIGroupV1Beta1
=== RUN   TestDiskAPIGroupV1Beta1/ListDiskIDs
--- FAIL: TestDiskAPIGroupV1Beta1 (0.00s)
    --- FAIL: TestDiskAPIGroupV1Beta1/ListDiskIDs (0.00s)
        disk_test.go:26:
                Error Trace:    disk_test.go:26
                Error:          Expected nil, but got: &status.statusError{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(nil)}, sizeCache:0, unknownFields:[]uint8(nil), Code:14, Message:"connection error: desc = \"transport: Error while dialing open \\\\\\\\.\\\\\\\\pipe\\\\\\\\csi-proxy-disk-v1beta1: The system cannot find the file specified.\"", Details:[]*anypb.Any(nil)}
                Test:           TestDiskAPIGroupV1Beta1/ListDiskIDs
=== RUN   TestFilesystemAPIGroup
=== RUN   TestFilesystemAPIGroup/PathExists_positive
=== RUN   TestFilesystemAPIGroup/IsMount
--- FAIL: TestFilesystemAPIGroup (0.01s)
    --- FAIL: TestFilesystemAPIGroup/PathExists_positive (0.00s)
        filesystem_test.go:45:
                Error Trace:    filesystem_test.go:45
                Error:          Expected nil, but got: &status.statusError{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(nil)}, sizeCache:0, unknownFields:[]uint8(nil), Code:14, Message:"connection error: desc = \"transport: Error while dialing open \\\\\\\\.\\\\\\\\pipe\\\\\\\\csi-proxy-filesystem-v1alpha1: The system cannot find the file specified.\"", Details:[]*anypb.Any(nil)}
                Test:           TestFilesystemAPIGroup/PathExists_positive
        filesystem_test.go:50:
                Error Trace:    filesystem_test.go:50
                Error:          Should be true
                Test:           TestFilesystemAPIGroup/PathExists_positive
                Messages:       <nil>
        filesystem_test.go:59:
                Error Trace:    filesystem_test.go:59
                Error:          Expected nil, but got: &status.statusError{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(nil)}, sizeCache:0, unknownFields:[]uint8(nil), Code:14, Message:"connection error: desc = \"transport: Error while dialing open \\\\\\\\.\\\\\\\\pipe\\\\\\\\csi-proxy-filesystem-v1alpha1: The system cannot find the file specified.\"", Details:[]*anypb.Any(nil)}
                Test:           TestFilesystemAPIGroup/PathExists_positive
        filesystem_test.go:67:
                Error Trace:    filesystem_test.go:67
                Error:          Expected nil, but got: &status.statusError{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(nil)}, sizeCache:0, unknownFields:[]uint8(nil), Code:14, Message:"connection error: desc = \"transport: Error while dialing open \\\\\\\\.\\\\\\\\pipe\\\\\\\\csi-proxy-filesystem-v1alpha1: The system cannot find the file specified.\"", Details:[]*anypb.Any(nil)}
                Test:           TestFilesystemAPIGroup/PathExists_positive
        filesystem_test.go:72:
                Error Trace:    filesystem_test.go:72
                Error:          Should be true
                Test:           TestFilesystemAPIGroup/PathExists_positive
                Messages:       <nil>
        filesystem_test.go:81:
                Error Trace:    filesystem_test.go:81
                Error:          Expected nil, but got: &status.statusError{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(nil)}, sizeCache:0, unknownFields:[]uint8(nil), Code:14, Message:"connection error: desc = \"transport: Error while dialing open \\\\\\\\.\\\\\\\\pipe\\\\\\\\csi-proxy-filesystem-v1alpha1: The system cannot find the file specified.\"", Details:[]*anypb.Any(nil)}
                Test:           TestFilesystemAPIGroup/PathExists_positive
        filesystem_test.go:95:
                Error Trace:    filesystem_test.go:95
                Error:          Expected nil, but got: &status.statusError{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(nil)}, sizeCache:0, unknownFields:[]uint8(nil), Code:14, Message:"connection error: desc = \"transport: Error while dialing open \\\\\\\\.\\\\\\\\pipe\\\\\\\\csi-proxy-filesystem-v1alpha1: The system cannot find the file specified.\"", Details:[]*anypb.Any(nil)}
                Test:           TestFilesystemAPIGroup/PathExists_positive
    --- FAIL: TestFilesystemAPIGroup/IsMount (0.00s)
        filesystem_test.go:123:
                Error Trace:    filesystem_test.go:123
                Error:          Expected nil, but got: &status.statusError{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(nil)}, sizeCache:0, unknownFields:[]uint8(nil), Code:14, Message:"connection error: desc = \"transport: Error while dialing open \\\\\\\\.\\\\\\\\pipe\\\\\\\\csi-proxy-filesystem-v1alpha1: The system cannot find the file specified.\"", Details:[]*anypb.Any(nil)}
                Test:           TestFilesystemAPIGroup/IsMount
=== RUN   TestSmbAPIGroup
=== RUN   TestSmbAPIGroup/Smb_positive
--- FAIL: TestSmbAPIGroup (0.57s)
    --- FAIL: TestSmbAPIGroup/Smb_positive (0.57s)
        smb_test.go:147: TestSmbAPIGroup setupUser failed: exit status 1, output: "New-Localuser : Access denied.\r\nAt line:1 char:67\r\n+ ... Text -Force;New-Localuser -name $Env:username -accountneverexpires -p ...\r\n+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : PermissionDenied: (geRcJ:LocalUser) [New-LocalUser], AccessDeniedException\r\n    + FullyQualifiedErrorId : AccessDenied,Microsoft.PowerShell.Commands.NewLocalUserCommand\r\n \r\n"
=== RUN   TestVolumeAPIs
=== RUN   TestVolumeAPIs/SimpleE2E
=== RUN   TestVolumeAPIs/NegativeDiskTests
=== RUN   TestVolumeAPIs/NegativeVolumeTests
--- FAIL: TestVolumeAPIs (0.67s)
    --- FAIL: TestVolumeAPIs/SimpleE2E (0.66s)
        volume_test.go:70: Error: exit status 1. Command: New-VHD -Path C:\var\lib\kubelet\plugins\testplugin-49.csi.io\disk-4.vhdx -SizeBytes 5368709120. Out: New-VHD : The term 'New-VHD' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was includ
            ed, verify that the path is correct and try again.
            At line:1 char:1
            + New-VHD -Path C:\var\lib\kubelet\plugins\testplugin-49.csi.io\disk-4. ...
            + ~~~~~~~
                + CategoryInfo          : ObjectNotFound: (New-VHD:String) [], CommandNotFoundException
                + FullyQualifiedErrorId : CommandNotFoundException

            .
        volume_test.go:27: Test failed. Skipping cleanup!
        volume_test.go:28: Mount path located at C:\var\lib\kubelet\plugins\testplugin-49.csi.io\mount-85
        volume_test.go:29: VHDx path located at C:\var\lib\kubelet\plugins\testplugin-49.csi.io\disk-4.vhdx
    --- PASS: TestVolumeAPIs/NegativeDiskTests (0.00s)
    --- PASS: TestVolumeAPIs/NegativeVolumeTests (0.00s)
FAIL
FAIL    github.com/kubernetes-csi/csi-proxy/integrationtests    19.414s
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/api/dummy        [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/api/dummy/v1     [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/api/dummy/v1alpha1       [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/api/dummy/v1alpha2       [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/client/dummy/v1  [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/client/dummy/v1alpha1    [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/client/dummy/v1alpha2    [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/server/dummy     [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/server/dummy/internal    [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/server/dummy/internal/v1 [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/server/dummy/internal/v1alpha1   [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/apigroups/server/dummy/internal/v1alpha2   [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/csiapigen/new_group/api    [no test files]
?       github.com/kubernetes-csi/csi-proxy/integrationtests/csiapigen/new_group/api/v1alpha1   [no test files]
FAIL

andyzhangx · 2020-09-08T03:55:17Z

let's enable all unit tests on Windows

ddebroy · 2020-09-14T06:49:46Z

/hold

ddebroy

csi-proxy.exe needs to be built and started first.

ddebroy · 2020-09-14T06:51:52Z

.github/workflows/windows.yml

@@ -26,3 +26,6 @@ jobs:
      - name: Run Windows Unit Tests
        run: |
          go test -v -race ./internal/...
+      - name: Run Windows Integration Tests
+        run: |
+          go test -v -race ./integrationtests/...


The integration tests expect csi-proxy binary to be built and running with the named pipes for the API groups listening. Otherwise, the tests would fail when trying to connect to the named pipes. Please build and start csi-proxy before executing the tests.

@ddebroy do you have guideline about how to do this? especially this: running with the named pipes for the API groups listening @mayankshah1607 could help while need your guide first, thanks.

There are a few projects depending on this csi-proxy, and if there is an example solution about how to run unit test on csi-proxy, that would be great.

@andyzhangx sorry about the delay in responding. The unit tests for csi-proxy are rather minimal as there isn't a lot of internal logic. I think the focus of this PR around integration tests (of the csi-proxy clients interacting with the underlying Windows APIs/cmdlets) would be ideal to enable. The only piece missing here is building and running csi-proxy before invoking go test -v -race ./integrationtests/... .

csi-proxy can be built and run using go build -a -o _output/csi-proxy.exe ./cmd/csi-proxy as the build action is already doing followed by starting csi-proxy.exe by simply starting the executable: csi-proxy.exe . Note that csi-proxy.exe will run indefinitely, so if github action has a way to start it as a background exe that will be great. Alternative would be to register and start csi-proxy.exe in service mode:

sc create csiproxy binPath= "C:\csiproxy\csi-proxy.exe --windows-service" net start csiproxy

We're running csi-proxy.exe in GH Actions as a background process in the csi-driver-smb repo - https://github.com/kubernetes-csi/csi-driver-smb/blob/master/.github/workflows/windows.yml#L28

Maybe we could do the same here as well?

@mayankshah1607 pls go ahead with another PR, I think there could be a lot more work to do besides github action config change. Thanks.

k8s-ci-robot · 2020-10-03T23:46:30Z

New changes are detected. LGTM label has been removed.

Add integration tests

k8s-ci-robot · 2020-10-16T19:05:05Z

@jingxu97: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ddebroy · 2020-10-19T05:57:13Z

This PR is superseded by #90. Closing this.

k8s-ci-robot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Aug 31, 2020

k8s-ci-robot requested review from ddebroy and PatrickLang August 31, 2020 17:50

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Aug 31, 2020

k8s-ci-robot assigned ddebroy Sep 1, 2020

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 1, 2020

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 14, 2020

ddebroy requested changes Sep 14, 2020

View reviewed changes

andyzhangx mentioned this pull request Sep 15, 2020

tests: fix unit tests' file path compatibility and error message assertion across Windows and Linux platforms kubernetes-sigs/azurefile-csi-driver#421

Merged

4 tasks

jingxu97 force-pushed the Aug/integrationtest branch from 9c681f4 to d889690 Compare October 3, 2020 23:46

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 3, 2020

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Oct 3, 2020

ddebroy mentioned this pull request Oct 6, 2020

Support CSI plugins in Windows kubernetes/enhancements#1122

Closed

jingxu97 force-pushed the Aug/integrationtest branch 4 times, most recently from b441b2a to 9954702 Compare October 7, 2020 18:02

Add integration tests on Git action

867c052

Add integration tests

jingxu97 force-pushed the Aug/integrationtest branch from 9954702 to 867c052 Compare October 7, 2020 18:11

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 16, 2020

ddebroy closed this Oct 19, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add integration tests on Git action #69

Add integration tests on Git action #69

jingxu97 commented Aug 31, 2020

k8s-ci-robot commented Aug 31, 2020

ddebroy commented Sep 1, 2020

k8s-ci-robot commented Sep 1, 2020

andyzhangx commented Sep 8, 2020

andyzhangx commented Sep 8, 2020

ddebroy commented Sep 14, 2020

ddebroy left a comment

ddebroy Sep 14, 2020

andyzhangx Sep 23, 2020

andyzhangx Sep 23, 2020

ddebroy Oct 2, 2020 •

edited

Loading

mayankshah1607 Oct 2, 2020 •

edited

Loading

andyzhangx Oct 7, 2020

k8s-ci-robot commented Oct 3, 2020

k8s-ci-robot commented Oct 16, 2020

ddebroy commented Oct 19, 2020

Add integration tests on Git action #69

Add integration tests on Git action #69

Conversation

jingxu97 commented Aug 31, 2020

k8s-ci-robot commented Aug 31, 2020

ddebroy commented Sep 1, 2020

k8s-ci-robot commented Sep 1, 2020

andyzhangx commented Sep 8, 2020

andyzhangx commented Sep 8, 2020

ddebroy commented Sep 14, 2020

ddebroy left a comment

Choose a reason for hiding this comment

ddebroy Sep 14, 2020

Choose a reason for hiding this comment

andyzhangx Sep 23, 2020

Choose a reason for hiding this comment

andyzhangx Sep 23, 2020

Choose a reason for hiding this comment

ddebroy Oct 2, 2020 • edited Loading

Choose a reason for hiding this comment

mayankshah1607 Oct 2, 2020 • edited Loading

Choose a reason for hiding this comment

andyzhangx Oct 7, 2020

Choose a reason for hiding this comment

k8s-ci-robot commented Oct 3, 2020

k8s-ci-robot commented Oct 16, 2020

ddebroy commented Oct 19, 2020

ddebroy Oct 2, 2020 •

edited

Loading

mayankshah1607 Oct 2, 2020 •

edited

Loading