From 1741956dfd7b7c2432ba5d32f7a721a4fac5c90a Mon Sep 17 00:00:00 2001 From: Tian Feng Date: Mon, 6 Nov 2023 09:37:11 -0800 Subject: [PATCH 1/6] docs: add docs for Sauce Orchestrate reporter --- docs/orchestrate/saucectl-configuration.md | 85 +++++++++++++++++----- 1 file changed, 66 insertions(+), 19 deletions(-) diff --git a/docs/orchestrate/saucectl-configuration.md b/docs/orchestrate/saucectl-configuration.md index 2a0e5aa25e..2771332cfd 100644 --- a/docs/orchestrate/saucectl-configuration.md +++ b/docs/orchestrate/saucectl-configuration.md @@ -84,25 +84,6 @@ saucectl run --ccy 5 --- -### `retries` - -

| OPTIONAL | INTEGER |

- -Sets the number of times to retry a failed suite. For more settings, you can refer to [passThreshold](#passThreshold). - -```yaml -sauce: - retries: 1 -``` - -Alternatively, you can override the file setting at runtime by setting the retries flag as an inline parameter of the `saucectl run` command: - -```bash -saucectl run --retries 1 -``` - ---- - ### `tunnel`

| OPTIONAL | OBJECT |

@@ -375,6 +356,72 @@ artifacts: --- +## `reporters` + +

| OPTIONAL | OBJECT |

+ +Configures additional reporting capabilities provided by `saucectl`. + +```yaml +reporters: + json: + enabled: true + filename: saucectl-report.json +``` +--- + +### `json` + +

| OPTIONAL | OBJECT |

+ +The JSON reporter creates a single report of all executed saucectl suites. + +```yaml +reporters: + json: + enabled: true + filename: saucectl-report.json + webhookURL: https://my-webhook-url +``` + +--- + +#### `enabled` + +

| OPTIONAL | BOOLEAN |

+ +Toggles the reporter on/off. + +```yaml + enabled: true +``` + +--- + +#### `webhookURL` + +

| OPTIONAL | STRING |

+ +Specifies the webhook URL. When saucectl test is finished, it'll send an HTTP POST with a JSON payload to the configured webhook URL. + +```yaml + webhookURL: https://my-webhook-url +``` + +--- + +#### `filename` + +

| OPTIONAL | STRING |

+ +Specifies the report filename. Defaults to "saucectl-report.json". + +```yaml + filename: my-saucectl-report.json +``` + +--- + ## saucectl Commands The following are some common `saucectl` commands for interacting with Sauce Orchestrate. From 358fe684748c5d1f74ad57a2ea0be27253c5ab27 Mon Sep 17 00:00:00 2001 From: Tian Feng Date: Mon, 6 Nov 2023 10:02:05 -0800 Subject: [PATCH 2/6] Update docs/orchestrate/saucectl-configuration.md Co-authored-by: Alex Plischke --- docs/orchestrate/saucectl-configuration.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/orchestrate/saucectl-configuration.md b/docs/orchestrate/saucectl-configuration.md index 2771332cfd..fc208b074b 100644 --- a/docs/orchestrate/saucectl-configuration.md +++ b/docs/orchestrate/saucectl-configuration.md @@ -393,6 +393,8 @@ reporters: Toggles the reporter on/off. ```yaml +reporters: + json: enabled: true ``` From c0375d4183f211d1344c0e83c39adf67e00f8254 Mon Sep 17 00:00:00 2001 From: Tian Feng Date: Mon, 6 Nov 2023 10:04:38 -0800 Subject: [PATCH 3/6] use entire config in example --- docs/orchestrate/saucectl-configuration.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/orchestrate/saucectl-configuration.md b/docs/orchestrate/saucectl-configuration.md index fc208b074b..28c8a917ed 100644 --- a/docs/orchestrate/saucectl-configuration.md +++ b/docs/orchestrate/saucectl-configuration.md @@ -407,6 +407,8 @@ reporters: Specifies the webhook URL. When saucectl test is finished, it'll send an HTTP POST with a JSON payload to the configured webhook URL. ```yaml +reporters: + json: webhookURL: https://my-webhook-url ``` @@ -419,6 +421,8 @@ Specifies the webhook URL. When saucectl test is finished, it'll send an HTTP PO Specifies the report filename. Defaults to "saucectl-report.json". ```yaml +reporters: + json: filename: my-saucectl-report.json ``` From f04909d26ea7531a1f19cd7db597fa5145ab4274 Mon Sep 17 00:00:00 2001 From: Tian Feng Date: Mon, 6 Nov 2023 11:01:06 -0800 Subject: [PATCH 4/6] example hygiene --- docs/orchestrate/saucectl-configuration.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/orchestrate/saucectl-configuration.md b/docs/orchestrate/saucectl-configuration.md index 28c8a917ed..57089ee086 100644 --- a/docs/orchestrate/saucectl-configuration.md +++ b/docs/orchestrate/saucectl-configuration.md @@ -151,6 +151,7 @@ The set of properties providing details about the test suites to run. May contai The name of the test suite, which will be reflected in the results and related artifacts. ```yaml +suites: - name: "saucy test" ``` @@ -163,7 +164,9 @@ The name of the test suite, which will be reflected in the results and related a Tell Sauce Orchestrate what kind of entrypoint process you are running. `workload` is important for security monitoring. Options are `webdriver` or `other`. For most configurations `webdriver` is required. ```yaml -workload: webdriver +suites: + - name: "saucy test" + workload: webdriver ``` --- From e36412724ff50ca0455f622cba7db9aa51712674 Mon Sep 17 00:00:00 2001 From: Tian Feng Date: Mon, 6 Nov 2023 11:05:59 -0800 Subject: [PATCH 5/6] example hygiene --- docs/orchestrate/saucectl-configuration.md | 30 ++++++++++++++-------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/docs/orchestrate/saucectl-configuration.md b/docs/orchestrate/saucectl-configuration.md index 57089ee086..2a98329210 100644 --- a/docs/orchestrate/saucectl-configuration.md +++ b/docs/orchestrate/saucectl-configuration.md @@ -192,7 +192,9 @@ suites: The location of your container image. Takes the format [registry]/[image]:[tag]. ```yaml -image: saucelabs/sl-demo-docker-primary:0.0.1 +suites: + - name: "saucy test" + image: saucelabs/sl-demo-docker-primary:0.0.1 ``` ### `imagePullAuth` @@ -202,9 +204,11 @@ image: saucelabs/sl-demo-docker-primary:0.0.1 The credentials needed to access an image hosted in a private registry. It is highly recommend to not hardcode credentials in your `saucectl` config. Use environment variables instead. ```yaml -imagePullAuth: - user: $DOCKER_USERNAME - token: $DOCKER_PASSWORD +suites: + - name: "saucy test" + imagePullAuth: + user: $DOCKER_USERNAME + token: $DOCKER_PASSWORD ``` ### `entrypoint` @@ -214,7 +218,9 @@ imagePullAuth: The command that is executed after the container is ready. ```yaml -entrypoint: mvn test +suites: + - name: "saucy test" + entrypoint: mvn test ``` ### `files` @@ -224,9 +230,11 @@ entrypoint: mvn test Files to be uploaded onto the container. Can be used for populating test data that your scripts access. src and dst must be an absolute path. ```yaml -files: - - src: "runsauce.json" - dst: "/workdir/runsauce.json" +suites: + - name: "saucy test" + files: + - src: "runsauce.json" + dst: "/workdir/runsauce.json" ``` ### `env` @@ -236,8 +244,10 @@ files: Environment variables to be injected into the container. Can be used for populating secrets used in your tests. These environment variables are not stored anywhere in Sauce Labs. ```yaml -env: - KEY: value +suites: + - name: "saucy test" + env: + KEY: value ``` :::note From d126a2fe462afca939789aeb624e6f2bad386fb9 Mon Sep 17 00:00:00 2001 From: Tian Feng Date: Mon, 6 Nov 2023 12:54:26 -0800 Subject: [PATCH 6/6] complete docker related config --- docs/orchestrate/saucectl-configuration.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/orchestrate/saucectl-configuration.md b/docs/orchestrate/saucectl-configuration.md index 2a98329210..642db8db1d 100644 --- a/docs/orchestrate/saucectl-configuration.md +++ b/docs/orchestrate/saucectl-configuration.md @@ -206,6 +206,7 @@ The credentials needed to access an image hosted in a private registry. It is hi ```yaml suites: - name: "saucy test" + image: saucelabs/sl-demo-docker-primary:0.0.1 imagePullAuth: user: $DOCKER_USERNAME token: $DOCKER_PASSWORD @@ -220,6 +221,7 @@ The command that is executed after the container is ready. ```yaml suites: - name: "saucy test" + image: saucelabs/sl-demo-docker-primary:0.0.1 entrypoint: mvn test ``` @@ -232,6 +234,7 @@ Files to be uploaded onto the container. Can be used for populating test data th ```yaml suites: - name: "saucy test" + image: saucelabs/sl-demo-docker-primary:0.0.1 files: - src: "runsauce.json" dst: "/workdir/runsauce.json"