Naming: rename setup and cleanup tasks

.github/workflows/actions.yml

@@ -187,7 +187,7 @@ jobs:
         #   sleep 1
         #done
         crystal build src/cnf-testsuite.cr 
-        ./cnf-testsuite setup 
+        ./cnf-testsuite install_dependencies 
 
         LOG_LEVEL=info crystal spec --warnings none --tag ${{ matrix.spec }} -v
 
@@ -392,9 +392,9 @@ jobs:
         mkdir /shared/$DIR
         mv cnf-testsuite /shared/$DIR 
         cd /shared/$DIR
-        ./cnf-testsuite setup
+        ./cnf-testsuite install_dependencies
         wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/${GITHUB_SHA}/example-cnfs/coredns/cnf-testsuite.yml
-        ./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+        ./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
         LOG_LEVEL=info ./cnf-testsuite all ~compatibility ~resilience ~reasonable_startup_time ~reasonable_image_size ~platform ~increase_capacity ~decrease_capacity ~install_script_helm ~helm_chart_valid ~helm_chart_published verbose
     - name: Delete Cluster
       if: ${{ always() }}
@@ -472,9 +472,9 @@ jobs:
         mkdir /shared/$DIR
         mv cnf-testsuite /shared/$DIR
         cd /shared/$DIR
-        ./cnf-testsuite setup
+        ./cnf-testsuite install_dependencies
         wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/${GITHUB_SHA}/example-cnfs/coredns/cnf-testsuite.yml
-        ./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+        ./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
         LOG_LEVEL=info ./cnf-testsuite all ~resilience ~compatibility ~pod_network_latency ~platform ~increase_capacity ~decrease_capacity ~liveness ~readiness ~rolling_update ~rolling_downgrade ~rolling_version_change ~nodeport_not_used ~hostport_not_used ~hardcoded_ip_addresses_in_k8s_runtime_configuration ~install_script_helm ~helm_chart_valid ~helm_chart_published ~rollback ~secrets_used ~immutable_configmap verbose
     - name: Delete Cluster
       if: ${{ always() }}
@@ -552,9 +552,9 @@ jobs:
         mkdir /shared/$DIR
         mv cnf-testsuite /shared/$DIR
         cd /shared/$DIR
-        ./cnf-testsuite setup
+        ./cnf-testsuite install_dependencies
         wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/${GITHUB_SHA}/example-cnfs/coredns/cnf-testsuite.yml
-        ./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+        ./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
         LOG_LEVEL=info ./cnf-testsuite all ~resilience ~platform ~liveness ~readiness ~rolling_update ~rolling_downgrade ~rolling_version_change ~nodeport_not_used ~hostport_not_used ~hardcoded_ip_addresses_in_k8s_runtime_configuration ~rollback ~secrets_used ~immutable_configmap ~reasonable_startup_time ~reasonable_image_size verbose
     - name: Delete Cluster
       if: ${{ always() }}

CNF_TESTSUITE_YML_USAGE.md

@@ -1,7 +1,7 @@
 # Test Suite Configuration Usage: cnf-testsuite.yml
 ### What is the cnf-testsuite.yml and why is it required?:
 
-The cnf-testsuite.yml is used by `cnf_setup` in order to install the CNF to be tested onto an existing K8s cluster. 
+The cnf-testsuite.yml is used by `cnf_install` in order to install the CNF to be tested onto an existing K8s cluster. 
 
 
 The information in the cnf-testsuite.yml is also used for additional configuration of some tests e.g. `white_list_container_names` is used for exculding containers from the [privileged_containers](https://github.com/cnti-testcatalog/testsuite/blob/main/src/tasks/workload/security.cr#L138) container test.
@@ -122,7 +122,7 @@ Described below: [link](#5G-parameters)
 
 Deployments are defined as three arrays, each for different installation method. Each array element represents one deployment, and they are meant to represent a single CNF together.
 At least one deployment should exist in CNF config for it to be a proper config.
-All info for deployments is used in cnf_setup and cnf_cleanup tasks.
+All info for deployments is used in cnf_install and cnf_uninstall tasks.
 
 ```yaml
 deployments:
@@ -134,6 +134,10 @@ deployments:
   ...
 ```
 
+##### Installation priority
+
+If deployments needed to be installed in specific order, installation_priority parameter should be used. All of the deployments would be installed in order from highest installation priority to lowest. This parameter also affects uninstallation order, it would be reversed: from lowest installation_priority to highest. If not specified, installation priority for deployment is 0.
+
 ##### helm_charts
 
 Deployment, defined by helm chart and helm repository.
@@ -143,6 +147,7 @@ Explanations with example:
 ```yaml
 helm_charts:
   - name: coredns  # Name of the deployment
+    installation_priority: 0  # Installation priority of deployment
     helm_repo_name: stable  # Name of the repository for the helm chart
     helm_repo_url: https://cncf.gitlab.io/stable  # Repository URL
     helm_chart_name: coredns  # Name of the helm chart in format repo_name/chart_name
@@ -157,6 +162,7 @@ Explanations with example:
 ```yaml
 helm_dirs:
   - name: envoy  # Name of the deployment
+    installation_priority: 0  # Installation priority of deployment
     helm_directory: chart  # Path to the directory with Chart.yaml, relative to CNF configuration file
     helm_values: --set myvalue=42 # Additional values that would be used for helm installation
     namespace: cnf-default # Namespace to which deployment would be installed (cnf-default is default)
@@ -169,6 +175,7 @@ Explanations with example:
 ```yaml
 manifests:
   - name: nginx  # Name of the deployment
+    installation_priority: 0  # Installation priority of deployment
     manifest_directory: manifests  # Path to the directory with deployment manifests, relative to CNF configuration file
 ```
 

INSTALL.md

@@ -147,10 +147,10 @@ export KUBECONFIG=path/to/mycluster.config
 
 *Note: We recommend running cnf-testsuite on a non-production cluster.*
 
-The next step is to run the `setup` which prepares the cnf-testsuite. This runs pre-reqs to verify you have everything needed in order to run the suite, simply run the following:
+The next step is to run the `install_dependencies` which prepares the cnf-testsuite. This runs pre-reqs to verify you have everything needed in order to run the suite, simply run the following:
 
 ```
-cnf-testsuite setup
+cnf-testsuite install_dependencies
 ```
 
 The test suite by default will pull docker images from https://docker.io. You can set your own username and password with local environment variables by doing the following:
@@ -177,7 +177,7 @@ source test.sh
 </details>
 
 ### Configuration
-Now cnf-testsuite is setup, we're ready to configure it to point at a CNF to test.
+Now cnf-testsuite is prepared, we're ready to configure it to point at a CNF to test.
 
 #### Using an Example CNF
 
@@ -188,7 +188,7 @@ wget -O cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/tes
 ```
 - The wget gets a working config file, now tell cnf-testsuite to use it by doing the following:
 ```
-cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
 ```
 
 - There are other examples in the [examples-cnfs](https://github.com/cnti-testcatalog/testsuite/tree/master/example-cnfs) directory that can be used for testing as well.
@@ -200,14 +200,14 @@ If you've brought your own CNF to test, review the [CNF_TESTSUITE_YML_USAGE.md](
 If you've followed the [CNF_TESTSUITE_YML_USAGE.md](CNF_TESTSUITE_YML_USAGE.md) guide and have your cnf-testsuite.yml ready, you can run the same command we ran for the example CNF to set it up:
 
 ```
-cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
 ```
 
 ### Running cnf-testsuite for the first time
 
 #### Running Tests
 
-If you want to run all tests, do the following (this is assuming your `cnf_setup` ran without errors in the [configuration](#Configuration) steps:)
+If you want to run all tests, do the following (this is assuming your `cnf_install` ran without errors in the [configuration](#Configuration) steps:)
 _For complete usage, see the [USAGE.md](USAGE.md) doc._
 
 ```
@@ -233,12 +233,12 @@ A test log file, eg. `cnf-testsuite-results-20201216.txt`, will be created which
 
 For more details on points, see our [POINTS.md](./POINTS.md) documentation.
 
-#### Cleaning Up
+#### Uninstallation
 
 Run the following to uninstall the CNF (this is assuming you installed the cnf-testsuite.yml in your present working directory):
 ```
-cnf-testsuite cnf_cleanup
+cnf-testsuite cnf_uninstall
 ```
-You can also run `cleanall` and cnf-testsuite will attempt to cleanup everything.
+You can also run `cleanup` and cnf-testsuite will attempt to cleanup everything.
 
-_NOTE: Cleanup does not handle manually deployed CNFs_
+_NOTE: CNF uninstallation does not handle manually deployed CNFs_

README.md

@@ -17,9 +17,9 @@ To get the CNTI Test Catalog up and running, see the [Installation Guide](INSTAL
 Prereqs: kubernetes cluster, wget, curl, helm 3.1.1 or greater on your system already.
 
 1. Install the latest test suite binary: `source <(curl -s https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/curl_install.sh)`
-2. Run `setup` to prepare the cnf-testsuite: `cnf-testsuite setup`
+2. Run `install_dependencies` to prepare the cnf-testsuite: `cnf-testsuite install_dependencies`
 3. Pull down an example CNF configuration to try: `curl -o cnf-testsuite.yml https://raw.githubusercontent.com/cnti-testcatalog/testsuite/main/example-cnfs/coredns/cnf-testsuite.yml`
-4. Initialize the test suite for using the CNF: `cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml`
+4. Initialize the test suite for using the CNF: `cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml`
 5. Run all of application/workload tests: `cnf-testsuite workload`
 
 #### More Usage docs

SOURCE_INSTALL.md

@@ -8,7 +8,7 @@ This INSTALL guide will detail the minimum requirements needed for cnf-testsuite
 
 - [**Pre-Requisites**](#Pre-Requisites)
 - [**Installation**](#Installation)
-- [**Setup**](#Setup)
+- [**Preparation**](#Preparation)
 - [**Configuration**](#Configuration)
 - [**Running cnf-testsuite for the first time**](#Running-cnf-testsuite-for-the-first-time)
 
@@ -149,23 +149,23 @@ crystal spec
 
 </p></details>
 
-### Setup
+### Preparation
 
-Now that we have a `cnf-testsuite` binary, we can run `setup` to ensure it has all the pre-requisites needed in order to successfully run tests and setup required installed_cnf_files/ directory and other files required for cnf-testsuite.
+Now that we have a `cnf-testsuite` binary, we can run `install_dependencies` to ensure it has all the pre-requisites needed in order to successfully run tests and prepare required installed_cnf_files/ directory and other files required for cnf-testsuite.
 
-- Run the following to setup cnf-testsuite:
+- Run the following to prepare cnf-testsuite:
   ```
-  ./cnf-testsuite setup
+  ./cnf-testsuite install_dependencies
   ```
 - If you have crystal installed, you can also run by:
   ```
-  crystal src/cnf-testsuite.cr setup
+  crystal src/cnf-testsuite.cr install_dependencies
   ```
   This should display output of all the pre-requisites (and install helm if not found on the system you intend to run from). Any missing requirements will need to be satisfied before proceeding or could result in errors, etc.
 
 ### Configuration
 
-Now that cnf-testsuite is installed and setup, we can now run CNF workloads and tests. We recommend installing and running a sample CNF to ensure cnf-testsuite is operational and set expectations of the output.
+Now that cnf-testsuite is installed and prepared, we can now run CNF workloads and tests. We recommend installing and running a sample CNF to ensure cnf-testsuite is operational and set expectations of the output.
 
 #### Configuring an example CNF
 
@@ -178,12 +178,12 @@ To use CoreDNS as an example CNF. Download the testsuite configuration to test C
 - Prepare the test suite to use the CNF by running:
   ```
   # via built binary
-  ./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+  ./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
   ```
   Or
   ```
   # via crystal
-  crystal src/cnf-testsuite.cr cnf_setup cnf-config=./cnf-testsuite.yml
+  crystal src/cnf-testsuite.cr cnf_install cnf-config=./cnf-testsuite.yml
   ```
 
 There are other examples in the [example cnfs](example-cnfs) folder if you would like to test others.
@@ -197,7 +197,7 @@ There are other examples in the [example cnfs](example-cnfs) folder if you would
 
 #### Running Tests
 
-If you want to run all tests for CoreDNS Example CNF, do the following (this is assuming your `cnf_setup` ran without errors in the [configuration](#Configuring-an-example-CNF) steps:)
+If you want to run all tests for CoreDNS Example CNF, do the following (this is assuming your `cnf_install` ran without errors in the [configuration](#Configuring-an-example-CNF) steps:)
 _For complete usage, see the [USAGE.md](USAGE.md) doc._
 
 ```
@@ -221,7 +221,7 @@ You can also run via `crystal` by replacing the `./cnf-testsuite` with `crystal
 #### More Example Usage (also see the [complete usage documentation](https://github.com/cnti-testcatalog/testsuite/blob/main/USAGE.md))
 
 ```
-# These assume you've already run the cnf_setup pointing at a cnf-testsuite.yml config above. You can always specify your config at the end of each command as well, eg:
+# These assume you've already run the cnf_install pointing at a cnf-testsuite.yml config above. You can always specify your config at the end of each command as well, eg:
 ./cnf-testsuite all cnf-config=<path to your config yml>/cnf-testsuite.yml
 
 # Runs all ga tests (generally available workload and platform tests)
@@ -253,17 +253,18 @@ A test log file, eg. `cnf-testsuite-results-20201216.txt`, will be created which
 
 For more details on points, see our [POINTS.md](./POINTS.md) documentation.
 
-#### Cleaning Up
+#### Uninstallation
 
 Run the following to uninstall the CNF:
 
 ```
-./cnf-testsuite cnf_cleanup
+./cnf-testsuite cnf_uninstall
 ```
 
-You can also run `cleanall` and cnf-testsuite will attempt to cleanup everything.
+You can also run `cleanup` and cnf-testsuite will attempt to cleanup everything.
+
+_NOTE: CNF uninstallation does not handle manually deployed CNFs_
 
-_NOTE: Cleanup does not handle manually deployed CNFs_
 #### NOTE: If the OpenMetrics version changes, the protobuf file will need to be regenerated
 ```
 git clone https://github.com/jeromegn/protobuf.cr

USAGE.md

@@ -62,12 +62,12 @@ crystal build src/cnf-testsuite.cr
 #### Installing a cnf:
 
 ```
-./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml
+./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml
 ```
 
 ##### Skip waiting for resource readiness during installation:
 ```
-./cnf-testsuite cnf_setup cnf-config=./cnf-testsuite.yml skip_wait_for_install
+./cnf-testsuite cnf_install cnf-config=./cnf-testsuite.yml skip_wait_for_install
 ```
 
 #### Running all of the platform and workload tests:

example-cnfs/coredns/README.md

@@ -18,13 +18,13 @@ Follow [Pre-req steps](../../INSTALL.md#pre-requisites), including
 Initialize the test suite
 
 ```
-./cnf-testsuite setup
+./cnf-testsuite install_dependencies
 ```
 
 Configure and deploy CoreDNS as the target CNF
 
 ```
-./cnf-testsuite cnf_setup cnf-path=example-cnfs/coredns/cnf-testsuite.yml
+./cnf-testsuite cnf_install cnf-path=example-cnfs/coredns/cnf-testsuite.yml
 ```
 
 Run the all the tests
@@ -38,5 +38,5 @@ Check the results file
 Uninstall the CNF (including undeployment of CoreDNS)
 
 ```
-./cnf-testsuite cnf_cleanup
+./cnf-testsuite cnf_uninstall
 ```

example-cnfs/envoy/README.md

@@ -12,16 +12,16 @@ Set the KUBECONFIG environment to point to the remote K8s cluster
 
 ### Automated Envoy installation
 
-Run cnf-testsuite setup
+Run cnf-testsuite install_dependencies
 
 ```
-./cnf-testsuite setup
+./cnf-testsuite install_dependencies
 ```
 
 Install Envoy
 
 ```
-./cnf-testsuite cnf_setup cnf-config=example-cnfs/envoy/cnf-testsuite.yml
+./cnf-testsuite cnf_install cnf-config=example-cnfs/envoy/cnf-testsuite.yml
 ```
 
 Run the test suite:
@@ -33,5 +33,5 @@ Run the test suite:
 Envoy uninstallation
 
 ```
-./cnf-testsuite cnf_cleanup
+./cnf-testsuite cnf_uninstall
 ```

example-cnfs/ip-forwarder/README.md

@@ -18,22 +18,22 @@ To run this example CNF, note the following requirements:
 ### Install helm version 3
 
 ### Automated installation
-Run cnf-testsuite setup 
+Run cnf-testsuite install_dependencies 
 ```
-export KUBECONFIG=$(pwd)/<YourKubeConf> ; ./cnf-testsuite setup
+export KUBECONFIG=$(pwd)/<YourKubeConf> ; ./cnf-testsuite install_dependencies
 ```
 
 Setup and deploy  service chain
 ```
-export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_setup cnf-path=example-cnfs/ip-forwarder/cnf-testsuite.yml deploy_with_chart=false
+export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_install cnf-path=example-cnfs/ip-forwarder/cnf-testsuite.yml deploy_with_chart=false
 ```
 
 ### Testing
 Run the test suite: `export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite all`
 
 ### Automated uninstallation
 ```
-export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_cleanup
+export KUBECONFIG=$(pwd)/admin.conf ; ./cnf-testsuite cnf_uninstall
 ```
 
 

example-cnfs/linkerd2/README.md

@@ -9,10 +9,10 @@ Set the KUBECONFIG environment to point to the remote K8s cluster
 
 ### Automated Linkerd installation
 
-Run cnf-testsuite setup
+Run cnf-testsuite install_dependencies
 
 ```
-./cnf-testsuite setup
+./cnf-testsuite install_dependencies
 ```
 
 Install linkerd
@@ -24,7 +24,7 @@ helm repo add linkerd https://helm.linkerd.io/stable
 
 helm install linkerd-crds linkerd/linkerd-crds -n linkerd --create-namespace 
 
-.cnf-testsuite cnf_setup cnf-path=example-cnfs/linkerd2/cnf-testsuite.yml
+.cnf-testsuite cnf_install cnf-path=example-cnfs/linkerd2/cnf-testsuite.yml
 ```
 
 Run the test suite:
@@ -36,5 +36,5 @@ Run the test suite:
 linkerd uninstallation
 
 ```
-./cnf-testsuite cnf_cleanup
+./cnf-testsuite cnf_uninstall
 ```