Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update contributor & testing guides #642

Merged
merged 10 commits into from
Jul 29, 2022
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@ build-cli: build-cli-linux-amd build-cli-linux-arm build-cli-mac-intel build-cli
build-injector-registry:
cd src/injector/stage2 && $(MAKE) build-bootstrap-registry

docs-and-schema:
go run main.go internal generate-cli-docs
.hooks/verify-zarf-schema.sh

# Inject and deploy a new dev version of zarf agent for testing (should have an existing zarf agent deployemt)
# @todo: find a clean way to support Kind or k3d: k3d image import $(tag)
dev-agent-image:
Expand Down
1 change: 0 additions & 1 deletion docs/4-user-guide/1-the-zarf-cli/100-cli-commands/zarf.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,3 @@ zarf [COMMAND] [flags]
* [zarf tools](zarf_tools.md) - Collection of additional tools to make airgap easier
* [zarf version](zarf_version.md) - Displays the version of the Zarf binary

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,3 @@ See each sub-command's help for details on how to use the generated script.
* [zarf completion powershell](zarf_completion_powershell.md) - Generate the autocompletion script for powershell
* [zarf completion zsh](zarf_completion_zsh.md) - Generate the autocompletion script for zsh

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,3 @@ zarf completion bash

* [zarf completion](zarf_completion.md) - Generate the autocompletion script for the specified shell

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,3 @@ zarf completion fish [flags]

* [zarf completion](zarf_completion.md) - Generate the autocompletion script for the specified shell

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,3 @@ zarf completion powershell [flags]

* [zarf completion](zarf_completion.md) - Generate the autocompletion script for the specified shell

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,3 @@ zarf completion zsh [flags]

* [zarf completion](zarf_completion.md) - Generate the autocompletion script for the specified shell

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Uses a k8s port-forward to connect to resources within the cluster referenced by
Three default options for this command are <REGISTRY|LOGGING|GIT>. These will connect to the Zarf created resources (assuming they were selected when performing the `zarf init` command).

Packages can provide service manifests that define their own shortcut connection options. These options will be printed to the terminal when the package finishes deploying.
If you don't remember what connection shortcuts your deployed package offers, you can search your cluster for services that have the 'zarf.dev/connect-name' label. The value of that label is the name you will pass into the 'zarf connect' command.
If you don't remember what connection shortcuts your deployed package offers, you can search your cluster for services that have the 'zarf.dev/connect-name' label. The value of that label is the name you will pass into the 'zarf connect' command.

Even if the packages you deploy don't define their own shortcut connection options, you can use the command flags to connect into specific resources. You can read the command flag descriptions below to get a better idea how to connect to whatever resource you are trying to connect to.

Expand Down Expand Up @@ -41,4 +41,3 @@ zarf connect <REGISTRY|LOGGING|GIT> [flags]
* [zarf](zarf.md) - DevSecOps Airgap Toolkit
* [zarf connect list](zarf_connect_list.md) - List all available connection shortcuts.

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,3 @@ zarf connect list [flags]

* [zarf connect](zarf_connect.md) - Access services or pods deployed in the cluster.

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -36,4 +36,3 @@ zarf destroy [flags]

* [zarf](zarf.md) - DevSecOps Airgap Toolkit

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Prepares a k8s cluster for the deployment of Zarf packages

### Synopsis

Injects a docker registry as well as other optional useful things (such as a git server and a logging stack) into a k8s cluster under the 'zarf' namespace to support future application deployments.
Injects a docker registry as well as other optional useful things (such as a git server and a logging stack) into a k8s cluster under the 'zarf' namespace to support future application deployments.
If you do not have a k8s cluster already configured, this command will give you the ability to install a cluster locally.

This command looks for a zarf-init package in the local directory that the command was executed from. If no package is found in the local directory and the Zarf CLI exists somewhere outside of the current directory, Zarf will failover and attempt to find a zarf-init package in the directory that the Zarf binary is located in.
Expand Down Expand Up @@ -38,4 +38,3 @@ zarf init [flags]

* [zarf](zarf.md) - DevSecOps Airgap Toolkit

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,3 @@ Zarf package commands for creating, deploying, and inspecting packages
* [zarf package deploy](zarf_package_deploy.md) - Use to deploy a Zarf package from a local file or URL (runs offline)
* [zarf package inspect](zarf_package_inspect.md) - Lists the payload of a Zarf package (runs offline)

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,3 @@ zarf package create [DIRECTORY] [flags]

* [zarf package](zarf_package.md) - Zarf package commands for creating, deploying, and inspecting packages

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -35,4 +35,3 @@ zarf package deploy [PACKAGE] [flags]

* [zarf package](zarf_package.md) - Zarf package commands for creating, deploying, and inspecting packages

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,3 @@ zarf package inspect [PACKAGE] [flags]

* [zarf package](zarf_package.md) - Zarf package commands for creating, deploying, and inspecting packages

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,7 @@ Tools to help prepare assets for packaging

* [zarf](zarf.md) - DevSecOps Airgap Toolkit
* [zarf prepare find-images](zarf_prepare_find-images.md) - Evaluates components in a zarf file to identify images specified in their helm charts and manifests
* [zarf prepare patch-git](zarf_prepare_patch-git.md) - Converts all .git URLs to the specified Zarf HOST and with the Zarf URL pattern in a given FILE. NOTE:
* [zarf prepare patch-git](zarf_prepare_patch-git.md) - Converts all .git URLs to the specified Zarf HOST and with the Zarf URL pattern in a given FILE. NOTE:
This should only be used for manifests that are not mutated by the Zarf Agent Mutating Webhook.
* [zarf prepare sha256sum](zarf_prepare_sha256sum.md) - Generate a SHA256SUM for the given file

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,3 @@ zarf prepare find-images [flags]

* [zarf prepare](zarf_prepare.md) - Tools to help prepare assets for packaging

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## zarf prepare patch-git

Converts all .git URLs to the specified Zarf HOST and with the Zarf URL pattern in a given FILE. NOTE:
Converts all .git URLs to the specified Zarf HOST and with the Zarf URL pattern in a given FILE. NOTE:
This should only be used for manifests that are not mutated by the Zarf Agent Mutating Webhook.

```
Expand All @@ -25,4 +25,3 @@ zarf prepare patch-git [HOST] [FILE] [flags]

* [zarf prepare](zarf_prepare.md) - Tools to help prepare assets for packaging

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,3 @@ zarf prepare sha256sum [FILE|URL] [flags]

* [zarf prepare](zarf_prepare.md) - Tools to help prepare assets for packaging

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,3 @@ Collection of additional tools to make airgap easier
* [zarf tools monitor](zarf_tools_monitor.md) - Launch K9s tool for managing K8s clusters
* [zarf tools registry](zarf_tools_registry.md) - Collection of registry commands provided by Crane

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,3 @@ Compress/Decompress tools for Zarf packages
* [zarf tools archiver compress](zarf_tools_archiver_compress.md) - Compress a collection of sources based off of the destination file extension
* [zarf tools archiver decompress](zarf_tools_archiver_decompress.md) - Decompress an archive (package) to a specified location.

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,3 @@ zarf tools archiver compress [SOURCES] [ARCHIVE] [flags]

* [zarf tools archiver](zarf_tools_archiver.md) - Compress/Decompress tools for Zarf packages

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,3 @@ zarf tools archiver decompress [ARCHIVE] [DESTINATION] [flags]

* [zarf tools archiver](zarf_tools_archiver.md) - Compress/Decompress tools for Zarf packages

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,3 @@ zarf tools get-admin-password [flags]

* [zarf tools](zarf_tools.md) - Collection of additional tools to make airgap easier

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,3 @@ zarf tools monitor [flags]

* [zarf tools](zarf_tools.md) - Collection of additional tools to make airgap easier

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,3 @@ Collection of registry commands provided by Crane
* [zarf tools registry pull](zarf_tools_registry_pull.md) - Pull remote images by reference and store their contents locally
* [zarf tools registry push](zarf_tools_registry_push.md) - Push local image contents to a remote registry

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,3 @@ zarf tools registry catalog [flags]

* [zarf tools registry](zarf_tools_registry.md) - Collection of registry commands provided by Crane

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,3 @@ zarf tools registry copy SRC DST [flags]

* [zarf tools registry](zarf_tools_registry.md) - Collection of registry commands provided by Crane

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,3 @@ zarf tools registry login [OPTIONS] [SERVER] [flags]

* [zarf tools registry](zarf_tools_registry.md) - Collection of registry commands provided by Crane

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ zarf tools registry pull IMAGE TARBALL [flags]
### Options

```
--annotate-ref Preserves image reference used to pull as an annotation when used with --format=oci
-c, --cache_path string Path to cache image layers
--format string Format in which to save images ("tarball", "legacy", or "oci") (default "tarball")
-h, --help help for pull
Expand All @@ -26,4 +27,3 @@ zarf tools registry pull IMAGE TARBALL [flags]

* [zarf tools registry](zarf_tools_registry.md) - Collection of registry commands provided by Crane

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,3 @@ zarf tools registry push PATH IMAGE [flags]

* [zarf tools registry](zarf_tools_registry.md) - Collection of registry commands provided by Crane

###### Auto generated by spf13/cobra on 25-Jul-2022
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,3 @@ zarf version [flags]

* [zarf](zarf.md) - DevSecOps Airgap Toolkit

###### Auto generated by spf13/cobra on 25-Jul-2022
14 changes: 6 additions & 8 deletions docs/6-developer-guide/1-contributor-guide.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
# Contributor Guide

:::caution Hard Hat Area
This page is still being developed. More content will be added soon!
:::

First off, thanks so much for wanting to help out! :tada:

The following is a set of guidelines for contributing to Zarf. These are mostly guidelines, not rules. Use your best judgement, and feel free to propose changes to this document in a pull request.
Expand All @@ -17,7 +13,9 @@ Specifically:
- We do trunk-based development with short-lived feature branches that originate from the trunk, get merged to the trunk, and are deleted after the merge
- We don't merge code into the trunk that isn't releasable
- We perform automated testing on all changes before they get merged to the trunk
- We squash feature branches on merge to keep the trunk pretty
- We create immutable release artifacts
- Every commit _must be signed_ by a core team member

### Developer Workflow

Expand All @@ -35,9 +33,9 @@ Here's what a typical "day in the life" of a Zarf developer might look like. Kee
1. Create a Draft Pull Request as soon as you are able to, even if it is just 5 minutes after you started working on it. Around here we aren't afraid to show unfinished work. It helps us get feedback more quickly.
1. :key: Create a Pull Request (or mark it Ready for Review if you've been working in a Draft PR).
1. :key: Automated tests will begin based on the paths you have edited in your Pull Request. More information on these tests can be found [here](https://docs.zarf.dev/docs/developer-guide/testing).
1. Clearly announce any breaking changes that have been made.
1. Clearly announce any breaking changes that have been made (include the [break-change](https://github.com/defenseunicorns/zarf/labels/breaking-change) label)
1. :key: Get at least 1 peer-review approval.
1. :key: Merge the PR into the trunk. We tend to prefer "Squash and Merge" but if your commits are on-point and you want to preserve them in the Git history of the trunk that's fine too.
1. :key: A team member will perform the merge after approval
1. Delete the branch
1. Close the issue if it got fully resolved by your PR. *Hint: You can add "Fixes #XX" to the PR description to automatically close an issue when the PR is merged.*

Expand Down Expand Up @@ -66,15 +64,15 @@ Our E2E tests can be found in the `/test` folder and follow the journey of someo

## Documentation

In this section you'll find documentation on documentation! Pun absolutely intended :smile:
PRs should also include any needed documentation updates. The cobra CLI docs and JSON schema docs are updated via `make docs-and-schema`.

### Architecture Decision Records (ADR)

We've chosen to use ADRs to document architecturally significant decisions. We primarily use the guidance found in [this article by Michael Nygard](http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions) with a couple of tweaks:

- The criteria for when an ADR is needed is undefined. The team will decide when the team needs an ADR.
- We will use the tool [adr-tools](https://github.com/npryce/adr-tools) to make it easier on ourselves to create and maintain ADRs.
- We will keep ADRs in the repository under `docs/adr/NNNN-name-of-adr.md`. `adr-tools` is configured with a dotfile to automatically use this directory and format.
- We will keep ADRs in the repository under `adr/NNNN-name-of-adr.md`. `adr-tools` is configured with a dotfile to automatically use this directory and format.

### How to use `adr-tools`

Expand Down
57 changes: 31 additions & 26 deletions docs/6-developer-guide/2-testing.md
Original file line number Diff line number Diff line change
@@ -1,56 +1,61 @@
# Code Testing

:::caution Hard Hat Area
This page is still being developed. More content will be added soon!
:::


Currently, we test Zarf through a series of end-to-end tests which can be found in the [e2e directory](https://github.com/defenseunicorns/zarf/tree/master/src/test/e2e) of the project. This directory holds all of our e2e tests that we use to verify Zarf functionality in an environment that replicates a live setting. The tests in this directory are automatically run against several K8s distros whenever a PR is opened or updated.

## Running Tests Locally

The tests in this directory are also able to be run locally!

### Dependencies
Running the tests locally have the same prerequisites as running and building Zarf:
1. GoLang >= `1.18.x`
2. Make
3. (for the `kind` and `k3d` options only) Docker
4. (for the `k3s` cluster only) Linux with root privileges

### Existing K8s Cluster
If you have a cluster already running, and use the env var `TESTDISTRO=provided`, the test suite will use the `KUBECONFIG` env var and the cluster that is currently configured as the active context. To make sure you are running in the right cluster, run something like `kubectl get nodes` with no optional flags and if the nodes that appear are the ones you expect, then the tests will use that cluster as well.

This means that you are able to run the tests against any remote distro you want, like EKS, AKS, GKE, RKE, etc.

### No Existing K8s Cluster
If you do not have a local cluster running, no worries! The e2e tests use the `sigs.k8s.io/kind` and `github.com/k3d-io/k3d/v5` libraries to stand up local clusters to test against. All you have to do is make sure Docker is running and set the `TESTDISTRO` env var to either `"kind"` or `"k3d"` and the test suite will automatically create the appropriate cluster before the test run, run the tests on it, then automatically destroy it to clean up.
Running the tests locally have the same prerequisites as running and building Zarf:
brandtkeller marked this conversation as resolved.
Show resolved Hide resolved

You can also use K3s by setting `TESTDISTRO=k3s` but note that there are extra requirements of being on Linux with root privileges.
1. GoLang >= `1.18.x`
2. Make
3. Any clean K8s cluster (local or remote) or Linux with sudo if you want to do the Zarf-installed K3s cluster

### Actually Running The Test

Here are a few different ways to run the tests, based on your specific situation:

```shell
# Note: You can prepend CI=true to these commands to force the --no-progress flag like CI does

# The default way, from the root directory of the repo. Will run all of the tests against your chosen k8s distro. Will automatically build any binary dependencies that don't already exist.
TESTDISTRO="[provided|kind|k3d|k3s]" make test-e2e
make test-e2e ARCH="[amd64|arm64]"

# To test against a Zarf-created cluster (on Linux with sudo)
APPLIANCE_MODE=true make test-e2e ARCH="[amd64|arm64]"

# If you already have everything build, you can run this inside this folder. This lets you customize the test run.
TESTDISTRO=YourChoiceHere go test ./... -v
go test ./... -v

# Let's say you only want to run one test. You would run:
TESTDISTRO=YourChoiceHere go test ./... -v -run TestFooBarBaz
test ./... -v -run TestFooBarBaz
```

:::note
The zarf binary and built packages need to live in the ./build directory but if you're trying to run the tests locally with 'go test ./...' then the zarf-init package will need to be in this directory.
:::

## Adding More Tests

There are a few requirements for all of our tests, that will need to be followed when new tests are added.

1. Tests may not run in parallel, since they use the same kubernetes cluster to run them.
2. Each test must begin with `defer e2e.cleanupAfterTest(t)` so that the cluster can be reset back to empty when finished.
2. Each test should begin with the entries below for standardization and test setup/teardown:

```go
t.Log("E2E: Enter useful description here")
e2e.setup(t)
defer e2e.teardown(t)
```

## Test Naming Conventions

## Coming Soon
1. More Linux distros tested
2. More K8s distros tested, including cloud distros like EKS
3. Make the tests that run in the CI pipeline more efficient by using more parallelization
The tests are run sequentially and the naming convention is set intentinonally:
- 00-19 tests run prior to `zarf init` (cluster not initialized)
- 20 is reserved for `zarf init`
- 21 is reserved for logging tests so they can be removed first (they take the most resources in the cluster)
- 22 is reserved for tests required the git-server, which is removed at the end of the test
- 23-99 are for the remaining tests that only require a basic zarf cluster without logging for the git-server
2 changes: 2 additions & 0 deletions src/cmd/internal.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ var generateCLIDocs = &cobra.Command{
Use: "generate-cli-docs",
Short: "Creates auto-generated markdown of all the commands for the CLI",
Run: func(cmd *cobra.Command, args []string) {
// Don't include the datestamp in the output
rootCmd.DisableAutoGenTag = true
//Generate markdown of the Zarf command (and all of its child commands)
doc.GenMarkdownTree(rootCmd, "./docs/4-user-guide/1-the-zarf-cli/100-cli-commands")
},
Expand Down