-
Notifications
You must be signed in to change notification settings - Fork 1.6k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Implement custom tester functionality in Skaffold (#5451)
* Adding new test runner for custom tester. * Adding new custom tester config to Skaffold comfig. * Adding usage example for newly added custon tester. * Extracting structure tests run logic into a seperate method. * Moving structure test's dependency file extraction logic to NewTester() to reduce code redundancy. * Revert "Moving structure test's dependency file extraction logic to NewTester() to reduce code redundancy." This reverts commit 5e1d859. * Moving structure test's dependency file extraction logic to NewTester() to reduce code redundancy. * Making the custom command a required field.[D * Adding new error codes for custom tester. * Updating custom test example. * Removing errors file. * Updating custom test example. * Adding generated schema files. * Added custom test example to `integration/run_test.go`. * Added custom test example in `integration/examples directory`. * Updating the apiVersion in example skaffold.yaml. * Updating the apiVersion in example skaffold.yaml. * Removing the custom-test form examples folder as the newly added custom test fields are not available in the current version of the skaffold config. * Updates per review feedback. * Added schema validation & updated example. * Adding tests for error scenarios. * Updated the test exit code. * Special casing error tests for Windows platform. * Updating the container name for custom test example. * Updated custom-test example container pods to continue running. * Added tests for TestDependencies. * Adding tests for custom test schema validation. * Updated unit tests per review feedback. * Updated unit tests to use mock by overriding util.RunCmd method. * Updating windows specific unit test. * Added another instance of custom command to the integration test. * Updated custom test dependencies. * Updating custom test example with command dependencies.
- Loading branch information
1 parent
7a04427
commit 2e55bec
Showing
15 changed files
with
794 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| FROM golang:1.15 as builder | ||
| COPY main.go . | ||
| # `skaffold debug` sets SKAFFOLD_GO_GCFLAGS to disable compiler optimizations | ||
| ARG SKAFFOLD_GO_GCFLAGS | ||
| RUN go build -gcflags="${SKAFFOLD_GO_GCFLAGS}" -o /app main.go | ||
|
|
||
| FROM alpine:3 | ||
| # Define GOTRACEBACK to mark this container as using the Go language runtime | ||
| # for `skaffold debug` (https://skaffold.dev/docs/workflows/debug/). | ||
| ENV GOTRACEBACK=single | ||
| CMD ["./app"] | ||
| COPY --from=builder /app . |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| ### Example: Running custom tests on built images | ||
|
|
||
| This example shows how to run _custom tests_ on newly built images in the skaffold dev loop. | ||
|
|
||
| Custom tests are associated with single image artifacts. When test dependencies change, no build will happen but tests would get re-run. Tests are configured in the `skaffold.yaml` in the `test` stanza, e.g. | ||
|
|
||
| ```yaml | ||
| test: | ||
| - image: skaffold-example | ||
| custom: | ||
| - command: <command> | ||
| timeoutSeconds: <timeout in seconds> | ||
| dependencies: <dependencies for this command> | ||
| paths: <file dependencies> | ||
| - <paths glob> | ||
| ``` | ||
| As tests take time, you might prefer to configure tests using [profiles](https://skaffold.dev/docs/https://skaffold.dev/docs/environment/profiles/) so that they can be automatically enabled or disabled, e.g. | ||
| If the `command` exits with a non-zero return code then the test will have failed, and deployment will not continue. | ||
|
|
||
| ```yaml | ||
| profiles: | ||
| - name: test | ||
| test: | ||
| - image: skaffold-example | ||
| custom: | ||
| - command: <command> | ||
| timeoutSeconds: <timeout in seconds> | ||
| dependencies: <dependencies for this command> | ||
| paths: <file dependencies> | ||
| - <paths glob> | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| apiVersion: v1 | ||
| kind: Pod | ||
| metadata: | ||
| name: custom-test | ||
| spec: | ||
| containers: | ||
| - name: custom-test | ||
| image: custom-test-example |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| /* | ||
| Copyright 2021 The Skaffold Authors | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|
|
||
| package main | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "math/rand" | ||
| "time" | ||
| ) | ||
|
|
||
| func MinInt(a, b int) int { | ||
| if a < b { | ||
| return a | ||
| } | ||
| return b | ||
| } | ||
|
|
||
| func main() { | ||
| rand.Seed(time.Now().UnixNano()) | ||
| for { | ||
| x := rand.Intn(100) | ||
| y := rand.Intn(100) | ||
|
|
||
| min := MinInt(x, y) | ||
| fmt.Println("Min of ", x, " and ", y, " is: ", min) | ||
| time.Sleep(time.Second * 1) | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| /* | ||
| Copyright 2021 The Skaffold Authors | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|
|
||
| package main | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "testing" | ||
| ) | ||
|
|
||
| func TestMinIntBasic(tb *testing.T) { | ||
| fmt.Println("Running Basic test.") | ||
| min := MinInt(5, -5) | ||
| if min != -5 { | ||
| tb.Errorf("MinInt(5, -5) returned %d; expecting -5", min) | ||
| } | ||
| } | ||
|
|
||
| func TestMinIntTableDriven(tdt *testing.T) { | ||
| var tests = []struct { | ||
| x, y int | ||
| want int | ||
| }{ | ||
| {0, 0, 0}, | ||
| {1, 0, 0}, | ||
| {0, 1, 0}, | ||
| {0, -1, -1}, | ||
| {-1, 0, -1}, | ||
| {-2, -5, -5}, | ||
| {-5, -2, -5}, | ||
| } | ||
|
|
||
| fmt.Println("Running Table driven test.") | ||
| for _, t := range tests { | ||
| testname := fmt.Sprintf("TestMinInt(): %d,%d", t.x, t.y) | ||
| tdt.Run(testname, func(tdt *testing.T) { | ||
| min := MinInt(t.x, t.y) | ||
| if min != t.want { | ||
| tdt.Errorf("MinInt(%d, %d) returned %d; expecting %d", t.x, t.y, min, t.want) | ||
| } | ||
| }) | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| apiVersion: skaffold/v2beta13 | ||
| kind: Config | ||
| build: | ||
| artifacts: | ||
| - image: custom-test-example | ||
| test: | ||
| - image: custom-test-example | ||
| custom: | ||
| - command: ./test.sh | ||
| timeoutSeconds: 60 | ||
| dependencies: | ||
| paths: | ||
| - "*_test.go" | ||
| - "test.sh" | ||
| - command: echo Hello world!! | ||
| dependencies: | ||
| command: echo [\"main_test.go\"] | ||
| deploy: | ||
| kubectl: | ||
| manifests: | ||
| - k8s-* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| #!/bin/bash | ||
|
|
||
| # Copyright 2021 The Skaffold Authors | ||
| # | ||
| # Licensed under the Apache License, Version 2.0 (the "License"); | ||
| # you may not use this file except in compliance with the License. | ||
| # You may obtain a copy of the License at | ||
| # | ||
| # http://www.apache.org/licenses/LICENSE-2.0 | ||
| # | ||
| # Unless required by applicable law or agreed to in writing, software | ||
| # distributed under the License is distributed on an "AS IS" BASIS, | ||
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| set -e | ||
|
|
||
| echo "go custom test $@" | ||
|
|
||
| go test . |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.