-
Notifications
You must be signed in to change notification settings - Fork 173
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
update contributor & testing guides (#642)
- Loading branch information
1 parent
fbda9ee
commit e25afd9
Showing
35 changed files
with
48 additions
and
69 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,56 +1,61 @@ | ||
# Code Testing | ||
|
||
:::caution Hard Hat Area | ||
This page is still being developed. More content will be added soon! | ||
::: | ||
|
||
|
||
Currently, we test Zarf through a series of end-to-end tests which can be found in the [e2e directory](https://github.com/defenseunicorns/zarf/tree/master/src/test/e2e) of the project. This directory holds all of our e2e tests that we use to verify Zarf functionality in an environment that replicates a live setting. The tests in this directory are automatically run against several K8s distros whenever a PR is opened or updated. | ||
|
||
## Running Tests Locally | ||
|
||
The tests in this directory are also able to be run locally! | ||
|
||
### Dependencies | ||
Running the tests locally have the same prerequisites as running and building Zarf: | ||
1. GoLang >= `1.18.x` | ||
2. Make | ||
3. (for the `kind` and `k3d` options only) Docker | ||
4. (for the `k3s` cluster only) Linux with root privileges | ||
|
||
### Existing K8s Cluster | ||
If you have a cluster already running, and use the env var `TESTDISTRO=provided`, the test suite will use the `KUBECONFIG` env var and the cluster that is currently configured as the active context. To make sure you are running in the right cluster, run something like `kubectl get nodes` with no optional flags and if the nodes that appear are the ones you expect, then the tests will use that cluster as well. | ||
|
||
This means that you are able to run the tests against any remote distro you want, like EKS, AKS, GKE, RKE, etc. | ||
|
||
### No Existing K8s Cluster | ||
If you do not have a local cluster running, no worries! The e2e tests use the `sigs.k8s.io/kind` and `github.com/k3d-io/k3d/v5` libraries to stand up local clusters to test against. All you have to do is make sure Docker is running and set the `TESTDISTRO` env var to either `"kind"` or `"k3d"` and the test suite will automatically create the appropriate cluster before the test run, run the tests on it, then automatically destroy it to clean up. | ||
Running the tests locally have the same prerequisites as running and building Zarf: | ||
|
||
You can also use K3s by setting `TESTDISTRO=k3s` but note that there are extra requirements of being on Linux with root privileges. | ||
1. GoLang >= `1.18.x` | ||
2. Make | ||
3. Any clean K8s cluster (local or remote) or Linux with sudo if you want to do the Zarf-installed K3s cluster | ||
|
||
### Actually Running The Test | ||
|
||
Here are a few different ways to run the tests, based on your specific situation: | ||
|
||
```shell | ||
# Note: You can prepend CI=true to these commands to force the --no-progress flag like CI does | ||
|
||
# The default way, from the root directory of the repo. Will run all of the tests against your chosen k8s distro. Will automatically build any binary dependencies that don't already exist. | ||
TESTDISTRO="[provided|kind|k3d|k3s]" make test-e2e | ||
make test-e2e ARCH="[amd64|arm64]" | ||
|
||
# To test against a Zarf-created cluster (on Linux with sudo) | ||
APPLIANCE_MODE=true make test-e2e ARCH="[amd64|arm64]" | ||
|
||
# If you already have everything build, you can run this inside this folder. This lets you customize the test run. | ||
TESTDISTRO=YourChoiceHere go test ./... -v | ||
go test ./... -v | ||
|
||
# Let's say you only want to run one test. You would run: | ||
TESTDISTRO=YourChoiceHere go test ./... -v -run TestFooBarBaz | ||
test ./... -v -run TestFooBarBaz | ||
``` | ||
|
||
:::note | ||
The zarf binary and built packages need to live in the ./build directory but if you're trying to run the tests locally with 'go test ./...' then the zarf-init package will need to be in this directory. | ||
::: | ||
|
||
## Adding More Tests | ||
|
||
There are a few requirements for all of our tests, that will need to be followed when new tests are added. | ||
|
||
1. Tests may not run in parallel, since they use the same kubernetes cluster to run them. | ||
2. Each test must begin with `defer e2e.cleanupAfterTest(t)` so that the cluster can be reset back to empty when finished. | ||
2. Each test should begin with the entries below for standardization and test setup/teardown: | ||
|
||
```go | ||
t.Log("E2E: Enter useful description here") | ||
e2e.setup(t) | ||
defer e2e.teardown(t) | ||
``` | ||
|
||
## Test Naming Conventions | ||
|
||
## Coming Soon | ||
1. More Linux distros tested | ||
2. More K8s distros tested, including cloud distros like EKS | ||
3. Make the tests that run in the CI pipeline more efficient by using more parallelization | ||
The tests are run sequentially and the naming convention is set intentinonally: | ||
- 00-19 tests run prior to `zarf init` (cluster not initialized) | ||
- 20 is reserved for `zarf init` | ||
- 21 is reserved for logging tests so they can be removed first (they take the most resources in the cluster) | ||
- 22 is reserved for tests required the git-server, which is removed at the end of the test | ||
- 23-99 are for the remaining tests that only require a basic zarf cluster without logging for the git-server |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters