From cd69c6997df6287f42cba46e474ee0b6353f94f4 Mon Sep 17 00:00:00 2001 From: Mike McDonald <61101829+mikemcd3912@users.noreply.github.com> Date: Wed, 8 Nov 2023 17:29:43 -0800 Subject: [PATCH 1/2] Update README.md Expanding test job requirements to cover repeatable testing using CronJobs --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 85b02ccb..926843fd 100644 --- a/README.md +++ b/README.md @@ -42,12 +42,12 @@ metadata: 2. Functional test should validate the functionality of the ISV product and describe what the ISV product does 3. Healthchecks, service endpoints checks or any other technical checks do not represent sufficient coverage required for the functional test 4. Functional test should be wrapped as a container, container image should be published on ECR, and/or provide evidence of successful recent vulnerability scan -5. Functional test must be implemented as a Kubernetes Job and any non-zero exit status of the job execution will be considered a failure -6. Functional test must be repeatable. That means that if the job has executed before successfully and no changes were applied, we expect to run it continuously and mark the product as failure if the test job starts producing failures even if previous executions against the same environment were successful +5. Functional test must be implemented as a Kubernetes Job or CronJob and any non-zero exit status of the job execution will be considered a failure +6. Functional test must be repeatable. That means that if the job has executed before successfully and no changes were applied, we expect to run it continuously and mark the product as failure if the test job starts producing failures even if previous executions against the same environment were successful. Functional test CronJobs should be configured to run at least once daily 7. Functional test should not require elevated security permissions, such as cluster roles, privileged mode, non-ephemeral storage 8. Functional test should be submitted under `eks-anywhere-common/testers` (runs on all platforms) or under your respective environment folder such as `eks-anywhere-snow/testers` (e.g. `eks-anywhere-snow/testers//`) -Refer the example [here](https://github.com/aws-samples/eks-anywhere-addons/tree/main/eks-anywhere-common/Testers/Hashicorp/Vault/kvJob.yaml) for functional test job. +Refer the example [here](https://github.com/aws-samples/eks-anywhere-addons/tree/main/eks-anywhere-common/Testers/Hashicorp/Vault/kvJob.yaml) for functional test job, or the example [here](https://github.com/aws-samples/eks-anywhere-addons/blob/main/eks-anywhere-common/Testers/Komodor/komodor-testjob.yaml) for functional test cronjob. ## Contribution Flow From e6ca9027e6143d2555c7353017cec911d6f6ade1 Mon Sep 17 00:00:00 2001 From: Mike McDonald <61101829+mikemcd3912@users.noreply.github.com> Date: Thu, 9 Nov 2023 14:02:26 -0800 Subject: [PATCH 2/2] Update README.md per Ela Merge Notes Removed references to Job and Job example for functional testing --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 926843fd..8ac726b8 100644 --- a/README.md +++ b/README.md @@ -42,12 +42,12 @@ metadata: 2. Functional test should validate the functionality of the ISV product and describe what the ISV product does 3. Healthchecks, service endpoints checks or any other technical checks do not represent sufficient coverage required for the functional test 4. Functional test should be wrapped as a container, container image should be published on ECR, and/or provide evidence of successful recent vulnerability scan -5. Functional test must be implemented as a Kubernetes Job or CronJob and any non-zero exit status of the job execution will be considered a failure +5. Functional test must be implemented as a Kubernetes CronJob and any non-zero exit status of the job execution will be considered a failure 6. Functional test must be repeatable. That means that if the job has executed before successfully and no changes were applied, we expect to run it continuously and mark the product as failure if the test job starts producing failures even if previous executions against the same environment were successful. Functional test CronJobs should be configured to run at least once daily 7. Functional test should not require elevated security permissions, such as cluster roles, privileged mode, non-ephemeral storage 8. Functional test should be submitted under `eks-anywhere-common/testers` (runs on all platforms) or under your respective environment folder such as `eks-anywhere-snow/testers` (e.g. `eks-anywhere-snow/testers//`) -Refer the example [here](https://github.com/aws-samples/eks-anywhere-addons/tree/main/eks-anywhere-common/Testers/Hashicorp/Vault/kvJob.yaml) for functional test job, or the example [here](https://github.com/aws-samples/eks-anywhere-addons/blob/main/eks-anywhere-common/Testers/Komodor/komodor-testjob.yaml) for functional test cronjob. +Refer the example [here](https://github.com/aws-samples/eks-anywhere-addons/blob/main/eks-anywhere-common/Testers/Komodor/komodor-testjob.yaml) for functional test cronjob. ## Contribution Flow