Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support running gRPC test cases #160

Merged
merged 20 commits into from
Aug 13, 2023
Merged

feat: support running gRPC test cases #160

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
78680fa
feat: add new fields to TestSuite for gRPC
Ink-33 Jul 26, 2023
386ad88
Merge remote-tracking branch 'origin/master' into grpc_test
Ink-33 Jul 26, 2023
342ee67
Merge remote-tracking branch 'origin/master' into grpc_test
Ink-33 Jul 31, 2023
0457608
feat: support gRPC test case
Ink-33 Aug 5, 2023
38560aa
Merge remote-tracking branch 'origin/master' into grpc_test
Ink-33 Aug 5, 2023
a5af73e
refactor: replace reflect.Value.Equal with reflect.DeepEqual
Ink-33 Aug 5, 2023
9d8b4b9
refactor: move verify out of runtestcase
Ink-33 Aug 13, 2023
a01f414
refactor: use factory to get runner
Ink-33 Aug 13, 2023
c418022
feat: new package pkg/compare
Ink-33 Aug 13, 2023
0edbcf5
refactor: use body field instead of payload
Ink-33 Aug 13, 2023
55faec8
refactor: new default testcase implement
Ink-33 Aug 13, 2023
11d7ca4
refactor: reduce code complexity
Ink-33 Aug 13, 2023
d9002bd
refactor: remove package errors
Ink-33 Aug 13, 2023
a9935ac
refactor: method of TestCaseRunner will not return itself
Ink-33 Aug 13, 2023
6ee92f7
test: fix unit test
Ink-33 Aug 13, 2023
1323c92
fix: error should be return in verify
Ink-33 Aug 13, 2023
2cef5bb
style: revert style change to yaml
Ink-33 Aug 13, 2023
73c1382
refactor: reduce code complexity
Ink-33 Aug 13, 2023
491d272
refactor: use runner factory
Ink-33 Aug 13, 2023
117fa07
license: add MIT header
Ink-33 Aug 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions cmd/run.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -252,9 +252,9 @@ func (o *runOption) runSuite(loader testing.Loader, dataContext map[string]inter
ctxWithTimeout, _ := context.WithTimeout(ctx, o.requestTimeout)
ctxWithTimeout = context.WithValue(ctxWithTimeout, runner.ContextKey("").ParentDir(), loader.GetContext())

simpleRunner := runner.NewSimpleTestCaseRunner()
simpleRunner.WithTestReporter(o.reporter)
if output, err = simpleRunner.RunTestCase(&testCase, dataContext, ctxWithTimeout); err != nil && !o.requestIgnoreError {
runner := getTestSuiteRunner(testSuite)
runner.WithTestReporter(o.reporter)
if output, err = runner.RunTestCase(&testCase, dataContext, ctxWithTimeout); err != nil && !o.requestIgnoreError {
err = fmt.Errorf("failed to run '%s', %v", testCase.Name, err)
return
} else {
Expand All @@ -269,3 +269,10 @@ func (o *runOption) runSuite(loader testing.Loader, dataContext map[string]inter
func getDefaultContext() map[string]interface{} {
return map[string]interface{}{}
}

func getTestSuiteRunner(suite *testing.TestSuite) runner.TestCaseRunner {
Ink-33 marked this conversation as resolved.
Show resolved Hide resolved
if suite.Spec.GRPC != nil {
return runner.NewGRPCTestCaseRunner(suite.API, *suite.Spec.GRPC)
}
return runner.NewSimpleTestCaseRunner()
}
47 changes: 42 additions & 5 deletions docs/api-testing-schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,38 @@
"properties": {
"kind": {
"type": "string",
"enum": ["openapi", "swagger"]
"enum": [
"openapi",
"swagger"
]
},
"url": {
"type": "string",
"qt-uri-protocols": [
"https", "http"
"https",
"http"
]
},
"grpc": {
"type": "object",
"additionalProperties": false,
"properties": {
"import": {
"type": "array",
"items": {
"type": "string"
}
},
"protofile": {
"type": "string"
},
"protoset": {
"type": "string"
},
"server_reflection": {
"type": "boolean"
}
}
}
}
},
Expand Down Expand Up @@ -91,6 +116,9 @@
"body": {
"type": "string"
},
"payload": {
"type": "string"
},
"header": {
"description": "HTTP response header",
"type": "object",
Expand Down Expand Up @@ -123,10 +151,16 @@
"qt-uri-protocols": [
"https"
]
},
},
"method": {
"type": "string",
"enum": ["GET", "POST", "PUT", "PATCH", "DELETE"]
"enum": [
"GET",
"POST",
"PUT",
"PATCH",
"DELETE"
]
},
"query": {
"description": "HTTP request query",
Expand All @@ -151,6 +185,9 @@
},
"bodyFromFile": {
"type": "string"
},
"payload": {
"type": "string"
}
},
"required": [
Expand All @@ -175,4 +212,4 @@
"title": "Job"
}
}
}
}
38 changes: 19 additions & 19 deletions extensions/collector/pkg/testdata/sample_suite.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,22 @@
# yaml-language-server: $schema=https://linuxsuren.github.io/api-testing/api-testing-schema.json
name: sample
items:
- name: v1
request:
api: http://foo/api/v1
method: GET
header:
Authorization: Bearer token
Content-Type: application/json
body: hello
- name: v1-1
request:
api: http://foo/api/v1
method: GET
header:
Authorization: Bearer token
Content-Type: application/json
body: hello
expect:
statusCode: 200
body: hello
- name: v1
request:
api: http://foo/api/v1
method: GET
header:
Authorization: Bearer token
Content-Type: application/json
body: hello
- name: v1-1
request:
api: http://foo/api/v1
method: GET
header:
Authorization: Bearer token
Content-Type: application/json
body: hello
expect:
statusCode: 200
body: hello
7 changes: 4 additions & 3 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ require (
github.com/Masterminds/sprig/v3 v3.2.3
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883
github.com/antonmedv/expr v1.12.1
github.com/bufbuild/protocompile v0.6.0
github.com/cucumber/godog v0.12.6
github.com/ghodss/yaml v1.0.0
github.com/grpc-ecosystem/grpc-gateway/v2 v2.16.0
Expand All @@ -14,12 +15,12 @@ require (
github.com/linuxsuren/go-fake-runtime v0.0.1
github.com/linuxsuren/unstructured v0.0.1
github.com/spf13/cobra v1.6.1
github.com/stretchr/testify v1.8.2
github.com/stretchr/testify v1.8.4
github.com/tidwall/gjson v1.14.4
github.com/xeipuuv/gojsonschema v1.2.0
golang.org/x/sync v0.1.0
golang.org/x/sync v0.3.0
google.golang.org/grpc v1.55.0
google.golang.org/protobuf v1.30.0
google.golang.org/protobuf v1.31.0
gopkg.in/yaml.v3 v3.0.1
)

Expand Down
15 changes: 8 additions & 7 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883 h1:bvNMNQO63//z+xNg
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/antonmedv/expr v1.12.1 h1:GTGrGN1kxxb+le0uQKaFRK8By4cvq1sleUCGE/U6hHg=
github.com/antonmedv/expr v1.12.1/go.mod h1:FPC8iWArxls7axbVLsW+kpg1mz29A1b2M6jt+hZfDkU=
github.com/bufbuild/protocompile v0.6.0 h1:Uu7WiSQ6Yj9DbkdnOe7U4mNKp58y9WDMKDn28/ZlunY=
github.com/bufbuild/protocompile v0.6.0/go.mod h1:YNP35qEYoYGme7QMtz5SBCoN4kL4g12jTtjuzRNdjpE=
github.com/cpuguy83/go-md2man/v2 v2.0.1/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
Expand Down Expand Up @@ -106,7 +108,6 @@ github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.3.1-0.20190311161405-34c6fa2dc709/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
Expand All @@ -115,8 +116,8 @@ github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/tidwall/gjson v1.14.4 h1:uo0p8EbA09J7RQaflQ1aBRffTR7xedD2bcIVSYxLnkM=
github.com/tidwall/gjson v1.14.4/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
Expand Down Expand Up @@ -145,8 +146,8 @@ golang.org/x/net v0.10.0 h1:X2//UzNDwYmtCLn7To6G58Wr6f5ahEAQgKNzv9Y951M=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.3.0 h1:ftCYgMx6zT/asHUrPw8BLLscYtGznsLAnjq5RH9P66E=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
Expand Down Expand Up @@ -179,8 +180,8 @@ google.golang.org/grpc v1.55.0 h1:3Oj82/tFSCeUrRTg/5E/7d/W5A1tj6Ky1ABAuZuv5ag=
google.golang.org/grpc v1.55.0/go.mod h1:iYEXKGkEBhg1PjZQvoYEVPTDkHo1/bjTnfwTeGONTY8=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.30.0 h1:kPPoIgf3TsEvrm0PFe15JQ+570QVxYzEvvHqChK+cng=
google.golang.org/protobuf v1.30.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
Expand Down
4 changes: 4 additions & 0 deletions go.work.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -746,6 +746,8 @@ github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa02
github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/linuxsuren/go-fake-runtime v0.0.0-20230426144714-1a7a0d160d3f h1:TfAzkLxq/agwMBbccTx/f/dlmFWIBLWRGCWjI4IOlK8=
github.com/linuxsuren/go-fake-runtime v0.0.0-20230426144714-1a7a0d160d3f/go.mod h1:zmh6J78hSnWZo68faMA2eKOdaEp8eFbERHi3ZB9xHCQ=
github.com/lyft/protoc-gen-star v0.6.0/go.mod h1:TGAoBVkt8w7MPG72TrKIu85MIdXwDuzJYeZuUPFPNwA=
github.com/lyft/protoc-gen-star v0.6.1/go.mod h1:TGAoBVkt8w7MPG72TrKIu85MIdXwDuzJYeZuUPFPNwA=
github.com/lyft/protoc-gen-star/v2 v2.0.1/go.mod h1:RcCdONR2ScXaYnQC5tUzxzlpA3WVYF7/opLeUgcQs/o=
Expand Down Expand Up @@ -936,6 +938,8 @@ golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220929204114-8fcdb60fdcc0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190312061237-fead79001313/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
Expand Down
Loading