Skip to content

Commit

Permalink
naming: rename setup and cleanup tasks
Browse files Browse the repository at this point in the history
Rename cnf_setup, cnf_cleanup and setup tasks to
cnf_install, cnf_uninstall, install_dependencies
Change documentation, specs and code accordingly

Refs: #2181
Signed-off-by: Konstantin Yarovoy <[email protected]>
  • Loading branch information
Konstantin Yarovoy committed Nov 29, 2024
1 parent 081c44b commit 07fab70
Show file tree
Hide file tree
Showing 63 changed files with 451 additions and 450 deletions.
14 changes: 7 additions & 7 deletions .github/workflows/actions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@ jobs:
echo "DOCKER USER RATE LIMIT: $AUTH_RATE_LIMIT"
crystal build src/cnf-testsuite.cr
./cnf-testsuite setup
./cnf-testsuite install_dependencies
LOG_LEVEL=info crystal spec --warnings none --tag ${{ matrix.spec }} -v
Expand Down Expand Up @@ -400,9 +400,9 @@ jobs:
mkdir /shared/$DIR
mv cnf-testsuite /shared/$DIR
cd /shared/$DIR
./cnf-testsuite setup
./cnf-testsuite install_dependencies
wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/${GITHUB_SHA}/example-cnfs/coredns/cnf-testsuite.yml
./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
LOG_LEVEL=info ./cnf-testsuite all ~compatibility ~resilience ~reasonable_startup_time ~reasonable_image_size ~platform ~increase_capacity ~decrease_capacity ~install_script_helm ~helm_chart_valid ~helm_chart_published verbose
- name: Delete Cluster
if: ${{ always() }}
Expand Down Expand Up @@ -475,9 +475,9 @@ jobs:
mkdir /shared/$DIR
mv cnf-testsuite /shared/$DIR
cd /shared/$DIR
./cnf-testsuite setup
./cnf-testsuite install_dependencies
wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/${GITHUB_SHA}/example-cnfs/coredns/cnf-testsuite.yml
./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
LOG_LEVEL=info ./cnf-testsuite all ~resilience ~compatibility ~pod_network_latency ~platform ~increase_capacity ~decrease_capacity ~liveness ~readiness ~rolling_update ~rolling_downgrade ~rolling_version_change ~nodeport_not_used ~hostport_not_used ~hardcoded_ip_addresses_in_k8s_runtime_configuration ~install_script_helm ~helm_chart_valid ~helm_chart_published ~rollback ~secrets_used ~immutable_configmap verbose
- name: Delete Cluster
if: ${{ always() }}
Expand Down Expand Up @@ -550,9 +550,9 @@ jobs:
mkdir /shared/$DIR
mv cnf-testsuite /shared/$DIR
cd /shared/$DIR
./cnf-testsuite setup
./cnf-testsuite install_dependencies
wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/${GITHUB_SHA}/example-cnfs/coredns/cnf-testsuite.yml
./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
LOG_LEVEL=info ./cnf-testsuite all ~resilience ~platform ~liveness ~readiness ~rolling_update ~rolling_downgrade ~rolling_version_change ~nodeport_not_used ~hostport_not_used ~hardcoded_ip_addresses_in_k8s_runtime_configuration ~rollback ~secrets_used ~immutable_configmap ~reasonable_startup_time ~reasonable_image_size verbose
- name: Delete Cluster
if: ${{ always() }}
Expand Down
4 changes: 2 additions & 2 deletions CNF_TESTSUITE_YML_USAGE.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Test Suite Configuration Usage: cnf-testsuite.yml
### What is the cnf-testsuite.yml and why is it required?:

The cnf-testsuite.yml is used by `cnf_setup` in order to install the CNF to be tested onto an existing K8s cluster.
The cnf-testsuite.yml is used by `cnf_install` in order to install the CNF to be tested onto an existing K8s cluster.


The information in the cnf-testsuite.yml is also used for additional configuration of some tests e.g. `white_list_container_names` is used for exculding containers from the [privileged_containers](https://github.com/cnti-testcatalog/testsuite/blob/main/src/tasks/workload/security.cr#L138) container test.
Expand Down Expand Up @@ -122,7 +122,7 @@ Described below: [link](#5G-parameters)

Deployments are defined as three arrays, each for different installation method. Each array element represents one deployment, and they are meant to represent a single CNF together.
At least one deployment should exist in CNF config for it to be a proper config.
All info for deployments is used in cnf_setup and cnf_cleanup tasks.
All info for deployments is used in cnf_install and cnf_uninstall tasks.

```yaml
deployments:
Expand Down
20 changes: 10 additions & 10 deletions INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,10 +147,10 @@ export KUBECONFIG=path/to/mycluster.config

*Note: We recommend running cnf-testsuite on a non-production cluster.*

The next step is to run the `setup` which prepares the cnf-testsuite. This runs pre-reqs to verify you have everything needed in order to run the suite, simply run the following:
The next step is to run the `install_dependencies` which prepares the cnf-testsuite. This runs pre-reqs to verify you have everything needed in order to run the suite, simply run the following:

```
cnf-testsuite setup
cnf-testsuite install_dependencies
```

The test suite by default will pull docker images from https://docker.io. You can set your own username and password with local environment variables by doing the following:
Expand All @@ -177,7 +177,7 @@ source test.sh
</details>

### Configuration
Now cnf-testsuite is setup, we're ready to configure it to point at a CNF to test.
Now cnf-testsuite is prepared, we're ready to configure it to point at a CNF to test.

#### Using an Example CNF

Expand All @@ -188,7 +188,7 @@ wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/tes
```
- The wget gets a working config file, now tell cnf-testsuite to use it by doing the following:
```
cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
```

- There are other examples in the [examples-cnfs](https://github.com/cnti-testcatalog/testsuite/tree/master/example-cnfs) directory that can be used for testing as well.
Expand All @@ -200,14 +200,14 @@ If you've brought your own CNF to test, review the [CNF_TESTSUITE_YML_USAGE.md](
If you've followed the [CNF_TESTSUITE_YML_USAGE.md](CNF_TESTSUITE_YML_USAGE.md) guide and have your cnf-testsuite.yml ready, you can run the same command we ran for the example CNF to set it up:

```
cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
```

### Running cnf-testsuite for the first time

#### Running Tests

If you want to run all tests, do the following (this is assuming your `cnf_setup` ran without errors in the [configuration](#Configuration) steps:)
If you want to run all tests, do the following (this is assuming your `cnf_install` ran without errors in the [configuration](#Configuration) steps:)
_For complete usage, see the [USAGE.md](USAGE.md) doc._

```
Expand All @@ -233,12 +233,12 @@ A test log file, eg. `cnf-testsuite-results-20201216.txt`, will be created which

For more details on points, see our [POINTS.md](./POINTS.md) documentation.

#### Cleaning Up
#### Uninstallation

Run the following to uninstall the CNF (this is assuming you installed the cnf-testsuite.yml in your present working directory):
```
cnf-testsuite cnf_cleanup
cnf-testsuite cnf_uninstall
```
You can also run `cleanall` and cnf-testsuite will attempt to cleanup everything.
You can also run `cleanup` and cnf-testsuite will attempt to cleanup everything.

_NOTE: Cleanup does not handle manually deployed CNFs_
_NOTE: CNF uninstallation does not handle manually deployed CNFs_
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ To get the CNTI Test Catalog up and running, see the [Installation Guide](INSTAL
Prereqs: kubernetes cluster, wget, curl, helm 3.1.1 or greater on your system already.

1. Install the latest test suite binary: `source <(curl -s https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/curl_install.sh)`
2. Run `setup` to prepare the cnf-testsuite: `cnf-testsuite setup`
2. Run `install_dependencies` to prepare the cnf-testsuite: `cnf-testsuite install_dependencies`
3. Pull down an example CNF configuration to try: `curl -o cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/example-cnfs/coredns/cnf-testsuite.yml`
4. Initialize the test suite for using the CNF: `cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml`
4. Initialize the test suite for using the CNF: `cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml`
5. Run all of application/workload tests: `cnf-testsuite workload`

#### More Usage docs
Expand Down
31 changes: 16 additions & 15 deletions SOURCE_INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This INSTALL guide will detail the minimum requirements needed for cnf-testsuite

- [**Pre-Requisites**](#Pre-Requisites)
- [**Installation**](#Installation)
- [**Setup**](#Setup)
- [**Preparation**](#Preparation)
- [**Configuration**](#Configuration)
- [**Running cnf-testsuite for the first time**](#Running-cnf-testsuite-for-the-first-time)

Expand Down Expand Up @@ -149,23 +149,23 @@ crystal spec

</p></details>

### Setup
### Preparation

Now that we have a `cnf-testsuite` binary, we can run `setup` to ensure it has all the pre-requisites needed in order to successfully run tests and setup required installed_cnf_files/ directory and other files required for cnf-testsuite.
Now that we have a `cnf-testsuite` binary, we can run `install_dependencies` to ensure it has all the pre-requisites needed in order to successfully run tests and prepare required installed_cnf_files/ directory and other files required for cnf-testsuite.

- Run the following to setup cnf-testsuite:
- Run the following to prepare cnf-testsuite:
```
./cnf-testsuite setup
./cnf-testsuite install_dependencies
```
- If you have crystal installed, you can also run by:
```
crystal src/cnf-testsuite.cr setup
crystal src/cnf-testsuite.cr install_dependencies
```
This should display output of all the pre-requisites (and install helm if not found on the system you intend to run from). Any missing requirements will need to be satisfied before proceeding or could result in errors, etc.

### Configuration

Now that cnf-testsuite is installed and setup, we can now run CNF workloads and tests. We recommend installing and running a sample CNF to ensure cnf-testsuite is operational and set expectations of the output.
Now that cnf-testsuite is installed and prepared, we can now run CNF workloads and tests. We recommend installing and running a sample CNF to ensure cnf-testsuite is operational and set expectations of the output.

#### Configuring an example CNF

Expand All @@ -178,12 +178,12 @@ To use CoreDNS as an example CNF. Download the testsuite configuration to test C
- Prepare the test suite to use the CNF by running:
```
# via built binary
./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
```
Or
```
# via crystal
crystal src/cnf-testsuite.cr cnf_setup cnf-config=./cnf-testsuite.yml
crystal src/cnf-testsuite.cr cnf_install cnf-config=./cnf-testsuite.yml
```

There are other examples in the [example cnfs](example-cnfs) folder if you would like to test others.
Expand All @@ -197,7 +197,7 @@ There are other examples in the [example cnfs](example-cnfs) folder if you would

#### Running Tests

If you want to run all tests for CoreDNS Example CNF, do the following (this is assuming your `cnf_setup` ran without errors in the [configuration](#Configuring-an-example-CNF) steps:)
If you want to run all tests for CoreDNS Example CNF, do the following (this is assuming your `cnf_install` ran without errors in the [configuration](#Configuring-an-example-CNF) steps:)
_For complete usage, see the [USAGE.md](USAGE.md) doc._

```
Expand All @@ -221,7 +221,7 @@ You can also run via `crystal` by replacing the `./cnf-testsuite` with `crystal
#### More Example Usage (also see the [complete usage documentation](https://github.com/cnti-testcatalog/testsuite/blob/main/USAGE.md))

```
# These assume you've already run the cnf_setup pointing at a cnf-testsuite.yml config above. You can always specify your config at the end of each command as well, eg:
# These assume you've already run the cnf_install pointing at a cnf-testsuite.yml config above. You can always specify your config at the end of each command as well, eg:
./cnf-testsuite all cnf-config=<path to your config yml>/cnf-testsuite.yml
# Runs all ga tests (generally available workload and platform tests)
Expand Down Expand Up @@ -253,17 +253,18 @@ A test log file, eg. `cnf-testsuite-results-20201216.txt`, will be created which

For more details on points, see our [POINTS.md](./POINTS.md) documentation.

#### Cleaning Up
#### Uninstallation

Run the following to uninstall the CNF:

```
./cnf-testsuite cnf_cleanup
./cnf-testsuite cnf_uninstall
```

You can also run `cleanall` and cnf-testsuite will attempt to cleanup everything.
You can also run `cleanup` and cnf-testsuite will attempt to cleanup everything.

_NOTE: CNF uninstallation does not handle manually deployed CNFs_

_NOTE: Cleanup does not handle manually deployed CNFs_
#### NOTE: If the OpenMetrics version changes, the protobuf file will need to be regenerated
```
git clone https://github.com/jeromegn/protobuf.cr
Expand Down
4 changes: 2 additions & 2 deletions USAGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,12 +62,12 @@ crystal build src/cnf-testsuite.cr
#### Installing a cnf:

```
./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
```

##### Skip waiting for resource readiness during installation:
```
./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml skip_wait_for_install
./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml skip_wait_for_install
```

#### Running all of the platform and workload tests:
Expand Down
6 changes: 3 additions & 3 deletions example-cnfs/coredns/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ Follow [Pre-req steps](../../INSTALL.md#pre-requisites), including
Initialize the test suite

```
./cnf-testsuite setup
./cnf-testsuite install_dependencies
```

Configure and deploy CoreDNS as the target CNF

```
./cnf-testsuite cnf_setup cnf-path=example-cnfs/coredns/cnf-testsuite.yml
./cnf-testsuite cnf_install cnf-path=example-cnfs/coredns/cnf-testsuite.yml
```

Run the all the tests
Expand All @@ -38,5 +38,5 @@ Check the results file
Uninstall the CNF (including undeployment of CoreDNS)

```
./cnf-testsuite cnf_cleanup
./cnf-testsuite cnf_uninstall
```
8 changes: 4 additions & 4 deletions example-cnfs/envoy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,16 @@ Set the KUBECONFIG environment to point to the remote K8s cluster

### Automated Envoy installation

Run cnf-testsuite setup
Run cnf-testsuite install_dependencies

```
./cnf-testsuite setup
./cnf-testsuite install_dependencies
```

Install Envoy

```
./cnf-testsuite cnf_setup cnf-config=example-cnfs/envoy/cnf-testsuite.yml
./cnf-testsuite cnf_install cnf-config=example-cnfs/envoy/cnf-testsuite.yml
```

Run the test suite:
Expand All @@ -33,5 +33,5 @@ Run the test suite:
Envoy uninstallation

```
./cnf-testsuite cnf_cleanup
./cnf-testsuite cnf_uninstall
```
8 changes: 4 additions & 4 deletions example-cnfs/ip-forwarder/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,22 +18,22 @@ To run this example CNF, note the following requirements:
### Install helm version 3

### Automated installation
Run cnf-testsuite setup
Run cnf-testsuite install_dependencies
```
export KUBECONFIG=$(pwd)/<YourKubeConf> ; ./cnf-testsuite setup
export KUBECONFIG=$(pwd)/<YourKubeConf> ; ./cnf-testsuite install_dependencies
```

Setup and deploy service chain
```
export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_setup cnf-path=example-cnfs/ip-forwarder/cnf-testsuite.yml deploy_with_chart=false
export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_install cnf-path=example-cnfs/ip-forwarder/cnf-testsuite.yml deploy_with_chart=false
```

### Testing
Run the test suite: `export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite all`

### Automated uninstallation
```
export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_cleanup
export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_uninstall
```


Expand Down
8 changes: 4 additions & 4 deletions example-cnfs/linkerd2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ Set the KUBECONFIG environment to point to the remote K8s cluster

### Automated Linkerd installation

Run cnf-testsuite setup
Run cnf-testsuite install_dependencies

```
./cnf-testsuite setup
./cnf-testsuite install_dependencies
```

Install linkerd
Expand All @@ -24,7 +24,7 @@ helm repo add linkerd https://helm.linkerd.io/stable
helm install linkerd-crds linkerd/linkerd-crds -n linkerd --create-namespace
.cnf-testsuite cnf_setup cnf-path=example-cnfs/linkerd2/cnf-testsuite.yml
.cnf-testsuite cnf_install cnf-path=example-cnfs/linkerd2/cnf-testsuite.yml
```

Run the test suite:
Expand All @@ -36,5 +36,5 @@ Run the test suite:
linkerd uninstallation

```
./cnf-testsuite cnf_cleanup
./cnf-testsuite cnf_uninstall
```
6 changes: 3 additions & 3 deletions example-cnfs/nsm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ Follow [Pre-req steps](../../INSTALL.md#pre-requisites), including
Initialize the test suite

```
./cnf-testsuite setup
./cnf-testsuite install_dependencies
```

Configure and deploy NSM as the target CNF

```
./cnf-testsuite cnf_setup cnf-config=./example-cnfs/nsm/cnf-testsuite.yml deploy_with_chart=false
./cnf-testsuite cnf_install cnf-config=./example-cnfs/nsm/cnf-testsuite.yml deploy_with_chart=false
```

Run the all the tests
Expand All @@ -34,5 +34,5 @@ Check the results file
Uninstall the CNF (including undeployment of NSM)

```
./cnf-testsuite cnf_cleanup
./cnf-testsuite cnf_uninstall
```
Loading

0 comments on commit 07fab70

Please sign in to comment.