From 2000c11b3c1c8af5164d30145a569fb284ef8c64 Mon Sep 17 00:00:00 2001 From: Yi Zha Date: Sun, 28 Jul 2024 17:33:46 +0800 Subject: [PATCH 01/20] doc: proposal for error message improvements Signed-off-by: Yi Zha --- docs/proposals/Error-Message-Improvements.md | 61 ++++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 docs/proposals/Error-Message-Improvements.md diff --git a/docs/proposals/Error-Message-Improvements.md b/docs/proposals/Error-Message-Improvements.md new file mode 100644 index 000000000..3b7d37fa5 --- /dev/null +++ b/docs/proposals/Error-Message-Improvements.md @@ -0,0 +1,61 @@ +# Error messages improvements + +## Problem/Motivation + +Error messages are crucial because they provide specific information about what went wrong, helping users quickly identify and resolve issues. Detailed error messages can pinpoint the exact part of the configuration or code that caused the problem. Error messages can include links or references to documentation, guiding users on how to fix the issue promptly. While testing the Ratify policy for signature verification, we noticed that some error messages were difficult to comprehend from the user perspective. + +Error message example 1: + +```text +time=2024-07-17T16:28:16.939576441Z level=warning msg=Original Error: (Original Error: (HEAD "https://roacr.azurecr.io/v2/net-monitor/manifests/v2": GET "https://roacr.azurecr.io/oauth2/token?scope=repository%3Anet-monitor%3Apull&service=roacr.azurecr.io": response status code 401: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information.), Error: repository operation failure, Code: REPOSITORY_OPERATION_FAILURE, Plugin Name: oras), Error: get subject descriptor failure, Code: GET_SUBJECT_DESCRIPTOR_FAILURE, Plugin Name: oras, Component Type: referrerStore, Detail: failed to resolve the subject descriptor component-type=referrerStore go.version=go1.21.10 namespace= trace-id=34b27888-5402-443e-9836-77124c840561 +``` + +The above example indicated the an error happened, however, a warning level was set for the log. It contained nested errors. The first original error message correctly described the source of the problem "401 unauthorized" and pointed to a document for resolution, however, errors following the original error were redundant and not well formatted, thus complicated the overall message. The overall message failed to describe the context of the error, although "401 unauthorized" explained the reason, but did this error happen during signature verification or else? What does the `subject` mean? + +Error message example 2: + +```text +"verifierReports": [ + { + subject: "docker.io/library/hello-world@sha256:1408fec50309afee38f3535383f5b09419e6dc0925bc69891e79d84cc4cdce6", + "isSuccess": false, + "message: "verification failed: Error: no verifier report, Code: NO_VERIFIER _REPORT, Component Type: executor, Description: No verifier report was generatec preventing access to the registry, or the absence of appropriate verifiers corresponding to the referenced image artifacts." + } +] +``` + +The json object `verifierReports` appears in the info level logs. It is structured and returned to the policy engine as external data inputs for policy decisions. If `isSuccess` field is set to `false`, the `message` field is set to error messages. In above example, the message is not well formatted and lacked clarity on the problem, its cause, and remediation methods. It's hard for users to understand in what context the error happened and what users need to do. For example, is it an error for signature verification? What does "no verifier report" mean? We also observed that the `verifierReports` contains different supported fields when compared with the Ratify Config policy and Ratify Rego policy, which is inconsistent. + +Error message example 3: + +```text +Error from server (Forbidden): admission webhook "validation.gatekeeper.sh" denied the request: [ratify-constraint] Subject failed verification: huishwabbit1.azurecr.io/test8may24@sha256:c780036bc8a6f577910bf01151013aaa18e255057a1653c76d8f3572aa3f6ff6 +``` + +The policy engine, for instance, Gatekeeper, has produced the above error message using a constraint template supplied by Ratify. It is the responsibility of the policy engine to tailor the constraint template for proper error messages to their requirements, however, it requires Ratify to provided useful verification reports as data inputs. In this example, the error message is not clear to users regarding the meaning of the term `Subject`, and fails to specify the context of failure, such as whether it was related to signature verification or SBOM verification or other verifications. Additionally, reasons behind the error were not provided. Furthermore, users may not be able to locate this error in the complete K8s logs to view more logs during error happened, because only artifact digest was shown and it is not enough to pinpoint the exact error in K8s logs. + +Further findings covering a range of cases such as KMP, Store, Verifier, Policy configuration, access control, and signature verification issues are recorded at https://hackmd.io/@H7a8_rG4SuaKwzu4NLT-9Q/rkMLwv1F0 (This link will be refreshed when the document is transferred to the Ratify repository.) + +In summary, the areas that need enhancement include: + +- Error messages similar to the first example will appear in Ratify logs. These may be found in the logs of Ratify Pods if Ratify is set up as a Kubernetes service, or they might be output by the Ratify CLI. The primary concerns include excessive nested errors, lacking error context, and not user-friendly error descriptions. +- Error messages contained within `verifierReport` that are sent back to the policy engine, as seen in the second example. They share issues similar to those mentioned for the first example. +- Ratify failed to provide sufficient information for the policy engine to generate error messages, which is demonstrated in the third example. + +The document aims to provide solutions and guidelines to improve error messages. + +## Scenarios + +### Error messages displayed in the Ratify logs + +Alice works as a DevOps engineer at Contoso. She set up tasks to deploy containerized apps into Kubernetes clusters. The cluster is assigned with the policy to deny the deployment of images that don't pass policy evaluation including verification of signature, SBOM, vulnerability reports and other image metadata. Alice knows that behind the scene, it is the Ratify conduct the verification and returned results as reports to the policy engine. When policy evaluation fails, Alice sees concise, clear and actionable error messages in Ratify logs. The error messages contain error descriptions, error reasons and mitigation solutions, allowing her to act on errors promptly. + +### Error messages displayed in verification reports used by the policy engine + +Bob is a software engineer on Contoso's Policy team, writing policies used during admissions in Kubernetes clusters. These policies evaluate images based on verifier reports generated by Ratify. If policy evaluation fails, Ratify sends back the reports with error messages to the policy engine. The reports, in JSON format, provide structured error messages that Bob utilizes to create clear and actionable error messages for the policy engine. These messages include descriptions, reasons for the errors, and mitigation solutions, allowing policy users to act on errors promptly. + +### Error messages returned by Ratify CLI commands + +Gina is a software engineer on the CI/CD team at Contoso, where she creates pipeline tasks incorporating Ratify CLI commands to assess artifacts according to policies. Should a policy check not pass, the corresponding artifacts are prevented from progressing in the pipeline. When policy evaluation fails, Gina sees concise, clear and actionable error messages returned by Ratify CLI commands. The error messages contain error descriptions, error reasons and mitigation solutions, allowing her to act on errors promptly. + +## Proposed solutions From 9dc9e8201f67b7bfb5c35d8b13c9fee4f163e84b Mon Sep 17 00:00:00 2001 From: Yi Zha Date: Sun, 28 Jul 2024 22:31:32 +0800 Subject: [PATCH 02/20] update Signed-off-by: Yi Zha --- ...ents.md => Error-Messages-Improvements.md} | 44 ++++++++++++++++--- 1 file changed, 39 insertions(+), 5 deletions(-) rename docs/proposals/{Error-Message-Improvements.md => Error-Messages-Improvements.md} (69%) diff --git a/docs/proposals/Error-Message-Improvements.md b/docs/proposals/Error-Messages-Improvements.md similarity index 69% rename from docs/proposals/Error-Message-Improvements.md rename to docs/proposals/Error-Messages-Improvements.md index 3b7d37fa5..eb67b62b7 100644 --- a/docs/proposals/Error-Message-Improvements.md +++ b/docs/proposals/Error-Messages-Improvements.md @@ -7,7 +7,14 @@ Error messages are crucial because they provide specific information about what Error message example 1: ```text -time=2024-07-17T16:28:16.939576441Z level=warning msg=Original Error: (Original Error: (HEAD "https://roacr.azurecr.io/v2/net-monitor/manifests/v2": GET "https://roacr.azurecr.io/oauth2/token?scope=repository%3Anet-monitor%3Apull&service=roacr.azurecr.io": response status code 401: unauthorized: authentication required, visit https://aka.ms/acr/authorization for more information.), Error: repository operation failure, Code: REPOSITORY_OPERATION_FAILURE, Plugin Name: oras), Error: get subject descriptor failure, Code: GET_SUBJECT_DESCRIPTOR_FAILURE, Plugin Name: oras, Component Type: referrerStore, Detail: failed to resolve the subject descriptor component-type=referrerStore go.version=go1.21.10 namespace= trace-id=34b27888-5402-443e-9836-77124c840561 +time=2024-07-17T16:28:16.939576441Z level=warning msg=Original Error: (Original Error: (HEAD "https:/ +roacr.azurecr.io/v2/net-monitor/manifests/v2": GET "https://roacr.azurecr.io/oauth2/token? +scope=repository%3Anet-monitor%3Apull&service=roacr.azurecr.io": response status code 401: unauthorized: +authentication required, visit https://aka.ms/acr/authorization for more information.), Error: repository +operation failure, Code: REPOSITORY_OPERATION_FAILURE, Plugin Name: oras), Error: get subject descriptor +failure, Code: GET_SUBJECT_DESCRIPTOR_FAILURE, Plugin Name: oras, Component Type: referrerStore, Detail: +failed to resolve the subject descriptor component-type=referrerStore go.version=go1.21.10 namespace= +trace-id=34b27888-5402-443e-9836-77124c840561 ``` The above example indicated the an error happened, however, a warning level was set for the log. It contained nested errors. The first original error message correctly described the source of the problem "401 unauthorized" and pointed to a document for resolution, however, errors following the original error were redundant and not well formatted, thus complicated the overall message. The overall message failed to describe the context of the error, although "401 unauthorized" explained the reason, but did this error happen during signature verification or else? What does the `subject` mean? @@ -29,7 +36,9 @@ The json object `verifierReports` appears in the info level logs. It is structur Error message example 3: ```text -Error from server (Forbidden): admission webhook "validation.gatekeeper.sh" denied the request: [ratify-constraint] Subject failed verification: huishwabbit1.azurecr.io/test8may24@sha256:c780036bc8a6f577910bf01151013aaa18e255057a1653c76d8f3572aa3f6ff6 +Error from server (Forbidden): admission webhook "validation.gatekeeper.sh" denied the request: +[ratify-constraint] Subject failed verification: huishwabbit1.azurecr.io/ +test8may24@sha256:c780036bc8a6f577910bf01151013aaa18e255057a1653c76d8f3572aa3f6ff6 ``` The policy engine, for instance, Gatekeeper, has produced the above error message using a constraint template supplied by Ratify. It is the responsibility of the policy engine to tailor the constraint template for proper error messages to their requirements, however, it requires Ratify to provided useful verification reports as data inputs. In this example, the error message is not clear to users regarding the meaning of the term `Subject`, and fails to specify the context of failure, such as whether it was related to signature verification or SBOM verification or other verifications. Additionally, reasons behind the error were not provided. Furthermore, users may not be able to locate this error in the complete K8s logs to view more logs during error happened, because only artifact digest was shown and it is not enough to pinpoint the exact error in K8s logs. @@ -48,14 +57,39 @@ The document aims to provide solutions and guidelines to improve error messages. ### Error messages displayed in the Ratify logs -Alice works as a DevOps engineer at Contoso. She set up tasks to deploy containerized apps into Kubernetes clusters. The cluster is assigned with the policy to deny the deployment of images that don't pass policy evaluation including verification of signature, SBOM, vulnerability reports and other image metadata. Alice knows that behind the scene, it is the Ratify conduct the verification and returned results as reports to the policy engine. When policy evaluation fails, Alice sees concise, clear and actionable error messages in Ratify logs. The error messages contain error descriptions, error reasons and mitigation solutions, allowing her to act on errors promptly. +Alice works as a DevOps engineer at Contoso. She set up tasks to deploy containerized apps into Kubernetes clusters. The cluster is assigned with the policy to deny the deployment of images that don't pass policy evaluation including verification of signature, SBOM, vulnerability reports and other image metadata. Alice knows that behind the scene, it is the Ratify conduct the verification and returned results as reports to the policy engine. When policy evaluation fails, Alice sees clear and actionable error messages in Ratify logs. The error messages contain concise error descriptions, error reasons and error recommendations, allowing her to act on errors promptly. ### Error messages displayed in verification reports used by the policy engine -Bob is a software engineer on Contoso's Policy team, writing policies used during admissions in Kubernetes clusters. These policies evaluate images based on verifier reports generated by Ratify. If policy evaluation fails, Ratify sends back the reports with error messages to the policy engine. The reports, in JSON format, provide structured error messages that Bob utilizes to create clear and actionable error messages for the policy engine. These messages include descriptions, reasons for the errors, and mitigation solutions, allowing policy users to act on errors promptly. +Bob is a software engineer on Contoso's Policy team, writing policies used during admissions in Kubernetes clusters. These policies evaluate images based on verifier reports generated by Ratify. If policy evaluation fails, Ratify sends back the reports with error messages to the policy engine. The reports, in JSON format, provide structured error messages that Bob utilizes to create clear and actionable error messages for the policy engine. These messages include concise error descriptions, error reasons, and error recommendations, allowing policy users to act on errors promptly. ### Error messages returned by Ratify CLI commands -Gina is a software engineer on the CI/CD team at Contoso, where she creates pipeline tasks incorporating Ratify CLI commands to assess artifacts according to policies. Should a policy check not pass, the corresponding artifacts are prevented from progressing in the pipeline. When policy evaluation fails, Gina sees concise, clear and actionable error messages returned by Ratify CLI commands. The error messages contain error descriptions, error reasons and mitigation solutions, allowing her to act on errors promptly. +Gina is a software engineer on the CI/CD team at Contoso, where she creates pipeline tasks incorporating Ratify CLI commands to assess artifacts according to policies. Should a policy check not pass, the corresponding artifacts are prevented from progressing in the pipeline. When policy evaluation fails, Gina sees concise, clear and actionable error messages returned by Ratify CLI commands. The error messages contain concise error descriptions, error reasons and error recommendations, allowing her to act on errors promptly. ## Proposed solutions + +[Azure CLI Error Handling Guidelines](https://github.com/Azure/azure-cli/blob/dev/doc/error_handling_guidelines.md#error-message) outlined a general pattern for error messages, consisting of: + +1. __What the error is.__ +2. __Why it happens.__ +3. __What users need to do to fix it.__ + +The proposed improvements for Ratify error messages adhere to this general pattern and the detailed DOs and DON'Ts provided in the guidelines. The recommended format for an error message is + +```text +[Error description]:[Error reason],[Error Recommendation] +``` + +So, for the above first example, the error message in the Ratify log can be improved to: + +```text +Failed to resolve the artifact descriptor: HEAD "https://roacr.azurecr.io/v2/net-monitor/manifests/v2": GET "https://roacr.azurecr.io/oauth2/token? +scope=repository%3Anet-monitor%3Apull&service=roacr.azurecr.io": response status code 401: unauthorized: +authentication required, visit https://aka.ms/acr/authorization for more information +``` + +## References + +- [Azure CLI Error Handling Guidelines](https://github.com/Azure/azure-cli/blob/dev/doc/error_handling_guidelines.md) +- [ORAS CLI Error Handling and Message Guideline](https://github.com/oras-project/oras/blob/v1.2.0/docs/proposals/error-handling-guideline.md) From 70ba627e21e076602377e579d3852672df867abc Mon Sep 17 00:00:00 2001 From: Yi Zha Date: Sun, 28 Jul 2024 22:33:00 +0800 Subject: [PATCH 03/20] update Signed-off-by: Yi Zha --- docs/proposals/Error-Messages-Improvements.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/proposals/Error-Messages-Improvements.md b/docs/proposals/Error-Messages-Improvements.md index eb67b62b7..5672dfbc2 100644 --- a/docs/proposals/Error-Messages-Improvements.md +++ b/docs/proposals/Error-Messages-Improvements.md @@ -26,7 +26,9 @@ Error message example 2: { subject: "docker.io/library/hello-world@sha256:1408fec50309afee38f3535383f5b09419e6dc0925bc69891e79d84cc4cdce6", "isSuccess": false, - "message: "verification failed: Error: no verifier report, Code: NO_VERIFIER _REPORT, Component Type: executor, Description: No verifier report was generatec preventing access to the registry, or the absence of appropriate verifiers corresponding to the referenced image artifacts." + "message: "verification failed: Error: no verifier report, Code: NO_VERIFIER _REPORT, Component Type: + executor, Description: No verifier report was generatec preventing access to the registry, or the absence + of appropriate verifiers corresponding to the referenced image artifacts." } ] ``` From 1d12f7faa81bfa181e571d32bec8386f0331ddf7 Mon Sep 17 00:00:00 2001 From: Yi Zha Date: Mon, 29 Jul 2024 17:04:35 +0800 Subject: [PATCH 04/20] update Signed-off-by: Yi Zha --- docs/proposals/Error-Messages-Improvements.md | 54 +++++++++++++++---- 1 file changed, 45 insertions(+), 9 deletions(-) diff --git a/docs/proposals/Error-Messages-Improvements.md b/docs/proposals/Error-Messages-Improvements.md index 5672dfbc2..e20a158fc 100644 --- a/docs/proposals/Error-Messages-Improvements.md +++ b/docs/proposals/Error-Messages-Improvements.md @@ -17,16 +17,16 @@ failed to resolve the subject descriptor component-type=referrerStore go.version trace-id=34b27888-5402-443e-9836-77124c840561 ``` -The above example indicated the an error happened, however, a warning level was set for the log. It contained nested errors. The first original error message correctly described the source of the problem "401 unauthorized" and pointed to a document for resolution, however, errors following the original error were redundant and not well formatted, thus complicated the overall message. The overall message failed to describe the context of the error, although "401 unauthorized" explained the reason, but did this error happen during signature verification or else? What does the `subject` mean? +The above example indicated the an error happened, however, a warning level was set for the log. The error message was set to the field `msg`. It contained nested errors. The first original error message correctly described the source of the problem "401 unauthorized" and pointed to a document for resolution, however, errors following the original error were redundant and not well formatted, thus complicated the overall message. The overall message failed to describe the context of the error, although "401 unauthorized" explained the reason, but did this error happen during signature verification or else? What does the `subject` mean? Error message example 2: ```text "verifierReports": [ { - subject: "docker.io/library/hello-world@sha256:1408fec50309afee38f3535383f5b09419e6dc0925bc69891e79d84cc4cdce6", + "subject": "docker.io/library/hello-world@sha256:1408fec50309afee38f3535383f5b09419e6dc0925bc69891e79d84cc4cdce6", "isSuccess": false, - "message: "verification failed: Error: no verifier report, Code: NO_VERIFIER _REPORT, Component Type: + "message": "verification failed: Error: no verifier report, Code: NO_VERIFIER _REPORT, Component Type: executor, Description: No verifier report was generatec preventing access to the registry, or the absence of appropriate verifiers corresponding to the referenced image artifacts." } @@ -71,24 +71,60 @@ Gina is a software engineer on the CI/CD team at Contoso, where she creates pipe ## Proposed solutions -[Azure CLI Error Handling Guidelines](https://github.com/Azure/azure-cli/blob/dev/doc/error_handling_guidelines.md#error-message) outlined a general pattern for error messages, consisting of: +We won’t create new error message guidelines; instead, we’ll refer to the existing ones. [Azure CLI Error Handling Guidelines](https://github.com/Azure/azure-cli/blob/dev/doc/error_handling_guidelines.md#error-message) outlined a general pattern for error messages, consisting of: 1. __What the error is.__ 2. __Why it happens.__ 3. __What users need to do to fix it.__ -The proposed improvements for Ratify error messages adhere to this general pattern and the detailed DOs and DON'Ts provided in the guidelines. The recommended format for an error message is +The proposed improvements for Ratify error messages adhere to this general pattern and the detailed DOs and DON'Ts provided in the guidelines. The error message will also include an error code. Since Ratify already supports a list of error codes, these can be used to search for remediation in the troubleshooting guide. For example, search error code `CERT_INVALID` in [the troubleshooting guide](https://ratify.dev/docs/troubleshoot/key-management-provider/kmp-tsg#cert_invalid). + +The recommended format for an error message in the Ratify log is as following. ```text -[Error description]:[Error reason],[Error Recommendation] +": : : " ``` -So, for the above first example, the error message in the Ratify log can be improved to: +For the error messages displayed in `verifierReports`, it is recommended to add two new optional fields `errorReason` and `remediation`, which will be used when the field `isSuccess` is set to `false`: + +```text +"verifierReports": [ + { + "subject": "" + "isSuccess": false, + "message": "", + "errorReason": "", + "remediation": "" + } +] +``` + +## Examples + +### Error messages displayed in the Ratify logs or returned by Ratify CLI commands + +For the above first example, the error message in the Ratify log can be improved to: ```text -Failed to resolve the artifact descriptor: HEAD "https://roacr.azurecr.io/v2/net-monitor/manifests/v2": GET "https://roacr.azurecr.io/oauth2/token? +REPOSITORY_OPERATION_FAILURE: Failed to resolve the artifact descriptor: HEAD "https://roacr.azurecr.io/v2/net-monitor/manifests/v2": GET "https://roacr.azurecr.io/oauth2/token? scope=repository%3Anet-monitor%3Apull&service=roacr.azurecr.io": response status code 401: unauthorized: -authentication required, visit https://aka.ms/acr/authorization for more information +authentication required, visit https://aka.ms/acr/authorization for more information. +``` + +### Error messages displayed in `verifierReports` + +For the second example, the error message can be improved to: + +```text +"verifierReports": [ + { + "subject": "docker.io/library/hello-world@sha256:1408fec50309afee38f3535383f5b09419e6dc0925bc69891e79d84cc4cdce6", + "isSuccess": false, + "message": "NO_VERIFIER_REPORT: Failed to verify artifact docker.io/library/hello-world@sha256:1408fec50309afee38f3535383f5b09419e6dc0925bc69891e79d84cc4cdce6: + "errorReason": "No signature verification report is found." + "remediation": "The artifact was either not signed and should not be trusted, signed but missing a Verifier configuration for verification, or needs to be signed if it should be." + } +] ``` ## References From 4ae43325eb72c733d40dc845e6f12f3d3db9931d Mon Sep 17 00:00:00 2001 From: Yi Zha Date: Mon, 29 Jul 2024 22:02:16 +0800 Subject: [PATCH 05/20] update Signed-off-by: Yi Zha --- docs/proposals/Error-Messages-Improvements.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/proposals/Error-Messages-Improvements.md b/docs/proposals/Error-Messages-Improvements.md index e20a158fc..74b7f8a89 100644 --- a/docs/proposals/Error-Messages-Improvements.md +++ b/docs/proposals/Error-Messages-Improvements.md @@ -45,7 +45,7 @@ test8may24@sha256:c780036bc8a6f577910bf01151013aaa18e255057a1653c76d8f3572aa3f6f The policy engine, for instance, Gatekeeper, has produced the above error message using a constraint template supplied by Ratify. It is the responsibility of the policy engine to tailor the constraint template for proper error messages to their requirements, however, it requires Ratify to provided useful verification reports as data inputs. In this example, the error message is not clear to users regarding the meaning of the term `Subject`, and fails to specify the context of failure, such as whether it was related to signature verification or SBOM verification or other verifications. Additionally, reasons behind the error were not provided. Furthermore, users may not be able to locate this error in the complete K8s logs to view more logs during error happened, because only artifact digest was shown and it is not enough to pinpoint the exact error in K8s logs. -Further findings covering a range of cases such as KMP, Store, Verifier, Policy configuration, access control, and signature verification issues are recorded at https://hackmd.io/@H7a8_rG4SuaKwzu4NLT-9Q/rkMLwv1F0 (This link will be refreshed when the document is transferred to the Ratify repository.) +Further findings covering a range of cases such as KMP, Store, Verifier, Policy configuration, access control, and signature verification issues are recorded at [Ratify Error Handling Scenarios.md](../discussion/Ratify%20Error%20Handling%20Scenarios.md) In summary, the areas that need enhancement include: From a4940093a6ff78e476e2e4dcb23d67e7dfdefee4 Mon Sep 17 00:00:00 2001 From: Yi Zha Date: Thu, 1 Aug 2024 17:25:37 +0800 Subject: [PATCH 06/20] update per comments Signed-off-by: Yi Zha --- docs/proposals/Error-Messages-Improvements.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/proposals/Error-Messages-Improvements.md b/docs/proposals/Error-Messages-Improvements.md index 74b7f8a89..cdaf73a06 100644 --- a/docs/proposals/Error-Messages-Improvements.md +++ b/docs/proposals/Error-Messages-Improvements.md @@ -33,7 +33,7 @@ Error message example 2: ] ``` -The json object `verifierReports` appears in the info level logs. It is structured and returned to the policy engine as external data inputs for policy decisions. If `isSuccess` field is set to `false`, the `message` field is set to error messages. In above example, the message is not well formatted and lacked clarity on the problem, its cause, and remediation methods. It's hard for users to understand in what context the error happened and what users need to do. For example, is it an error for signature verification? What does "no verifier report" mean? We also observed that the `verifierReports` contains different supported fields when compared with the Ratify Config policy and Ratify Rego policy, which is inconsistent. +When Ratify completes artifact verification, the result is returned to the policy engine in the format of the json object `verifierReports`. The `verifierReports` is also recorded in an INFO log of Ratify. If `isSuccess` field is set to `false`, the `message` field is set to error messages. In above example, the message is not well formatted and lacked clarity on the problem, its cause, and remediation methods. It's hard for users to understand in what context the error happened and what users need to do. For example, is it an error for signature verification? What does "no verifier report" mean? We also observed that the `verifierReports` contains different supported fields when compared with the Ratify Config policy and Ratify Rego policy, which is inconsistent. Error message example 3: @@ -45,12 +45,12 @@ test8may24@sha256:c780036bc8a6f577910bf01151013aaa18e255057a1653c76d8f3572aa3f6f The policy engine, for instance, Gatekeeper, has produced the above error message using a constraint template supplied by Ratify. It is the responsibility of the policy engine to tailor the constraint template for proper error messages to their requirements, however, it requires Ratify to provided useful verification reports as data inputs. In this example, the error message is not clear to users regarding the meaning of the term `Subject`, and fails to specify the context of failure, such as whether it was related to signature verification or SBOM verification or other verifications. Additionally, reasons behind the error were not provided. Furthermore, users may not be able to locate this error in the complete K8s logs to view more logs during error happened, because only artifact digest was shown and it is not enough to pinpoint the exact error in K8s logs. -Further findings covering a range of cases such as KMP, Store, Verifier, Policy configuration, access control, and signature verification issues are recorded at [Ratify Error Handling Scenarios.md](../discussion/Ratify%20Error%20Handling%20Scenarios.md) +Further findings covering a range of cases such as Key Management Provider (KMP), Store, Verifier, Policy configuration, access control, and signature verification issues are recorded at [Ratify Error Handling Scenarios.md](../discussion/Ratify%20Error%20Handling%20Scenarios.md) In summary, the areas that need enhancement include: - Error messages similar to the first example will appear in Ratify logs. These may be found in the logs of Ratify Pods if Ratify is set up as a Kubernetes service, or they might be output by the Ratify CLI. The primary concerns include excessive nested errors, lacking error context, and not user-friendly error descriptions. -- Error messages contained within `verifierReport` that are sent back to the policy engine, as seen in the second example. They share issues similar to those mentioned for the first example. +- Error messages contained within `verifierReports` that are sent back to the policy engine, as seen in the second example. These error messages share issues similar to those mentioned for the first example. The policy engine can customize their Rego policies using the messages inside the `verifierReports` to return to users in the logs or UI of the policy engine. - Ratify failed to provide sufficient information for the policy engine to generate error messages, which is demonstrated in the third example. The document aims to provide solutions and guidelines to improve error messages. @@ -121,8 +121,8 @@ For the second example, the error message can be improved to: "subject": "docker.io/library/hello-world@sha256:1408fec50309afee38f3535383f5b09419e6dc0925bc69891e79d84cc4cdce6", "isSuccess": false, "message": "NO_VERIFIER_REPORT: Failed to verify artifact docker.io/library/hello-world@sha256:1408fec50309afee38f3535383f5b09419e6dc0925bc69891e79d84cc4cdce6: - "errorReason": "No signature verification report is found." - "remediation": "The artifact was either not signed and should not be trusted, signed but missing a Verifier configuration for verification, or needs to be signed if it should be." + "errorReason": "No signature is found or wrong configuration" + "remediation": "Please either sign the artifact or set up signature verification for it." } ] ``` From 1401080a3099db5a707cfb95aeed17452b6c5e34 Mon Sep 17 00:00:00 2001 From: Yi Zha Date: Fri, 2 Aug 2024 17:23:34 +0800 Subject: [PATCH 07/20] update per comments Signed-off-by: Yi Zha --- docs/proposals/Error-Messages-Improvements.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/proposals/Error-Messages-Improvements.md b/docs/proposals/Error-Messages-Improvements.md index cdaf73a06..d01b54e06 100644 --- a/docs/proposals/Error-Messages-Improvements.md +++ b/docs/proposals/Error-Messages-Improvements.md @@ -122,11 +122,13 @@ For the second example, the error message can be improved to: "isSuccess": false, "message": "NO_VERIFIER_REPORT: Failed to verify artifact docker.io/library/hello-world@sha256:1408fec50309afee38f3535383f5b09419e6dc0925bc69891e79d84cc4cdce6: "errorReason": "No signature is found or wrong configuration" - "remediation": "Please either sign the artifact or set up signature verification for it." + "remediation": "Please either sign the artifact or configure verifiers for signature verification. Learn more at https://ratify.dev/docs/plugins/verifier/notation." } ] ``` +> This link https://ratify.dev/docs/plugins/verifier/notation is used as an example to illustrate the improvements. The link should vary depending on the particular error encountered. + ## References - [Azure CLI Error Handling Guidelines](https://github.com/Azure/azure-cli/blob/dev/doc/error_handling_guidelines.md) From 8b170533d0699ff8a780274e9adf8a8a331ef6cd Mon Sep 17 00:00:00 2001 From: Binbin Li Date: Thu, 25 Jul 2024 15:47:01 +0000 Subject: [PATCH 08/20] feat: add verifierName, verifierType and errorReason fields to verifierReport --- httpserver/types.go | 6 ++-- httpserver/types_test.go | 4 +-- pkg/executor/core/executor.go | 23 +++++++++------ pkg/verifier/api.go | 21 ++++++++----- pkg/verifier/cosign/cosign.go | 35 +++++++++++++--------- pkg/verifier/cosign/cosign_test.go | 47 +++++++++++++++++++----------- pkg/verifier/notation/notation.go | 12 ++++---- pkg/verifier/types/types.go | 36 ++++++++++++++--------- 8 files changed, 113 insertions(+), 71 deletions(-) diff --git a/httpserver/types.go b/httpserver/types.go index f307bc55d..8e3655cb5 100644 --- a/httpserver/types.go +++ b/httpserver/types.go @@ -22,9 +22,11 @@ import ( const ( VerificationResultVersion = "0.1.0" + ResultVersion0_2_0 = "0.2.0" // Starting from this version, the verification result can be // evaluated by Ratify embedded OPA engine. ResultVersionSupportingRego = "1.0.0" + ResultVersion1_1_0 = "1.1.0" ) type VerificationResponse struct { @@ -34,9 +36,9 @@ type VerificationResponse struct { } func fromVerifyResult(res types.VerifyResult, policyType string) VerificationResponse { - version := VerificationResultVersion + version := ResultVersion0_2_0 if policyType == pt.RegoPolicy { - version = ResultVersionSupportingRego + version = ResultVersion1_1_0 } return VerificationResponse{ Version: version, diff --git a/httpserver/types_test.go b/httpserver/types_test.go index ce2c359a2..04ed6da5c 100644 --- a/httpserver/types_test.go +++ b/httpserver/types_test.go @@ -32,12 +32,12 @@ func TestFromVerifyResult(t *testing.T) { { name: "Rego policy", policyType: pt.RegoPolicy, - expectedVersion: "1.0.0", + expectedVersion: "1.1.0", }, { name: "Config policy", policyType: pt.ConfigPolicy, - expectedVersion: "0.1.0", + expectedVersion: "0.2.0", }, } diff --git a/pkg/executor/core/executor.go b/pkg/executor/core/executor.go index ef9d27210..2cc90b028 100644 --- a/pkg/executor/core/executor.go +++ b/pkg/executor/core/executor.go @@ -175,19 +175,22 @@ func (executor Executor) verifyReferenceForJSONPolicy(ctx context.Context, subje if verifier.CanVerify(ctx, referenceDesc) { verifierStartTime := time.Now() verifyResult, err := verifier.Verify(ctx, subjectRef, referenceDesc, referrerStore) - verifyResult.Subject = subjectRef.String() if err != nil { verifyResult = vr.VerifierResult{ - IsSuccess: false, - Name: verifier.Name(), - Type: verifier.Type(), - Message: errors.ErrorCodeVerifyReferenceFailure.NewError(errors.Verifier, verifier.Name(), errors.EmptyLink, err, nil, errors.HideStackTrace).Error()} + IsSuccess: false, + Name: verifier.Name(), + Type: verifier.Type(), + VerifierName: verifier.Name(), + VerifierType: verifier.Type(), + Message: errors.ErrorCodeVerifyReferenceFailure.NewError(errors.Verifier, verifier.Name(), errors.EmptyLink, err, nil, errors.HideStackTrace).Error()} } if len(verifier.GetNestedReferences()) > 0 { executor.addNestedVerifierResult(ctx, referenceDesc, subjectRef, &verifyResult) } + verifyResult.Subject = subjectRef.String() + verifyResult.ReferenceDigest = referenceDesc.Digest.String() verifyResult.ArtifactType = referenceDesc.ArtifactType verifyResults = append(verifyResults, verifyResult) isSuccess = verifyResult.IsSuccess @@ -227,10 +230,12 @@ func (executor Executor) verifyReferenceForRegoPolicy(ctx context.Context, subje verifierResult, err := verifier.Verify(errCtx, subjectRef, referenceDesc, referrerStore) if err != nil { verifierReport = vt.VerifierResult{ - IsSuccess: false, - Name: verifier.Name(), - Type: verifier.Type(), - Message: errors.ErrorCodeVerifyReferenceFailure.NewError(errors.Verifier, verifier.Name(), errors.EmptyLink, err, nil, errors.HideStackTrace).Error()} + IsSuccess: false, + Name: verifier.Name(), // Deprecating Name in next release, reference to VerifierName instead. + Type: verifier.Type(), // Deprecating Type in next release, reference to VerifierType instead. + VerifierName: verifier.Name(), + VerifierType: verifier.Type(), + Message: errors.ErrorCodeVerifyReferenceFailure.NewError(errors.Verifier, verifier.Name(), errors.EmptyLink, err, nil, errors.HideStackTrace).Error()} } else { verifierReport = vt.NewVerifierResult(verifierResult) } diff --git a/pkg/verifier/api.go b/pkg/verifier/api.go index dcc549662..8f9695515 100644 --- a/pkg/verifier/api.go +++ b/pkg/verifier/api.go @@ -25,14 +25,19 @@ import ( // VerifierResult describes the result of verifying a reference manifest for a subject type VerifierResult struct { //nolint:revive // ignore linter to have unique type name - Subject string `json:"subject,omitempty"` - IsSuccess bool `json:"isSuccess"` - Name string `json:"name,omitempty"` - Type string `json:"type,omitempty"` - Message string `json:"message,omitempty"` - Extensions interface{} `json:"extensions,omitempty"` - NestedResults []VerifierResult `json:"nestedResults,omitempty"` - ArtifactType string `json:"artifactType,omitempty"` + Subject string `json:"subject,omitempty"` + IsSuccess bool `json:"isSuccess"` + Name string `json:"name,omitempty"` + VerifierName string `json:"verifierName,omitempty"` + Type string `json:"type,omitempty"` + VerifierType string `json:"verifierType,omitempty"` + Message string `json:"message,omitempty"` + ErrorReason string `json:"errorReason,omitempty"` + Extensions interface{} `json:"extensions,omitempty"` + NestedResults []VerifierResult `json:"nestedResults,omitempty"` + ArtifactType string `json:"artifactType,omitempty"` + ReferenceDigest string `json:"referenceDigest,omitempty"` + Remediation string `json:"remediation,omitempty"` } // ReferenceVerifier is an interface that defines methods to verify a reference diff --git a/pkg/verifier/cosign/cosign.go b/pkg/verifier/cosign/cosign.go index 40a67fbfb..a5a3bc4c4 100644 --- a/pkg/verifier/cosign/cosign.go +++ b/pkg/verifier/cosign/cosign.go @@ -285,11 +285,13 @@ func (v *cosignVerifier) verifyInternal(ctx context.Context, subjectReference co if hasValidSignature { return verifier.VerifierResult{ - Name: v.name, - Type: v.verifierType, - IsSuccess: true, - Message: "cosign verification success. valid signatures found. please refer to extensions field for verifications performed.", - Extensions: Extension{SignatureExtension: sigExtensions, TrustPolicy: trustPolicy.GetName()}, + Name: v.name, + Type: v.verifierType, + VerifierName: v.name, + VerifierType: v.verifierType, + IsSuccess: true, + Message: "Verification success. Valid signatures found. Please refer to extensions field for verifications performed.", + Extensions: Extension{SignatureExtension: sigExtensions, TrustPolicy: trustPolicy.GetName()}, }, nil } @@ -396,11 +398,13 @@ func (v *cosignVerifier) verifyLegacy(ctx context.Context, subjectReference comm if len(signatures) > 0 { return verifier.VerifierResult{ - Name: v.name, - Type: v.verifierType, - IsSuccess: true, - Message: "cosign verification success. valid signatures found", - Extensions: LegacyExtension{SignatureExtension: sigExtensions}, + Name: v.name, + Type: v.verifierType, + VerifierName: v.name, + VerifierType: v.verifierType, + IsSuccess: true, + Message: "Verification success. Valid signatures found", + Extensions: LegacyExtension{SignatureExtension: sigExtensions}, }, nil } @@ -482,10 +486,13 @@ func staticLayerOpts(desc imgspec.Descriptor) ([]static.Option, error) { // ErrorToVerifyResult returns a verifier result with the error message and isSuccess set to false func errorToVerifyResult(name string, verifierType string, err error) verifier.VerifierResult { return verifier.VerifierResult{ - IsSuccess: false, - Name: name, - Type: verifierType, - Message: errors.Wrap(err, "cosign verification failed").Error(), + IsSuccess: false, + Name: name, + Type: verifierType, + VerifierName: name, + VerifierType: verifierType, + Message: "Verification failed", + ErrorReason: err.Error(), } } diff --git a/pkg/verifier/cosign/cosign_test.go b/pkg/verifier/cosign/cosign_test.go index 4ceb03a96..fde624a3e 100644 --- a/pkg/verifier/cosign/cosign_test.go +++ b/pkg/verifier/cosign/cosign_test.go @@ -407,8 +407,8 @@ func TestErrorToVerifyResult(t *testing.T) { if verifierResult.Type != "cosign" { t.Errorf("errorToVerifyResult() = %v, want %v", verifierResult.Type, "cosign") } - if verifierResult.Message != "cosign verification failed: test error" { - t.Errorf("errorToVerifyResult() = %v, want %v", verifierResult.Message, "cosign verification failed: test error") + if verifierResult.Message != "Verification failed" { + t.Errorf("errorToVerifyResult() = %v, want %v", verifierResult.Message, "Verification failed") } } @@ -562,6 +562,7 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry cosignOpts cosign.CheckOpts store *mocks.MemoryTestStore expectedResultMessagePrefix string + expectedErrorReason string defaultCosignOpts bool }{ { @@ -569,14 +570,16 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry keys: map[PKKey]keymanagementprovider.PublicKey{}, getKeysError: true, store: &mocks.MemoryTestStore{}, - expectedResultMessagePrefix: "cosign verification failed: error", + expectedResultMessagePrefix: "Verification failed", + expectedErrorReason: "error", }, { name: "manifest fetch error", keys: map[PKKey]keymanagementprovider.PublicKey{}, getKeysError: false, store: &mocks.MemoryTestStore{}, - expectedResultMessagePrefix: "cosign verification failed: failed to get reference manifest", + expectedResultMessagePrefix: "Verification failed", + expectedErrorReason: "failed to get reference manifest: manifest not found", }, { name: "incorrect reference manifest media type error", @@ -589,7 +592,8 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry }, }, }, - expectedResultMessagePrefix: "cosign verification failed: reference manifest is not an image", + expectedResultMessagePrefix: "Verification failed", + expectedErrorReason: "reference manifest is not an image", }, { name: "failed subject descriptor fetch", @@ -602,7 +606,8 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry }, }, }, - expectedResultMessagePrefix: "cosign verification failed: failed to create subject hash", + expectedResultMessagePrefix: "Verification failed", + expectedErrorReason: "failed to create subject hash: subject not found for sha256:5678", }, { name: "failed to fetch blob", @@ -628,7 +633,8 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry }, }, }, - expectedResultMessagePrefix: "cosign verification failed: failed to get blob content", + expectedResultMessagePrefix: "Verification failed", + expectedErrorReason: "failed to get blob content: blob not found", }, { name: "invalid key type for AKV", @@ -659,7 +665,8 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry blobDigest: validSignatureBlob, }, }, - expectedResultMessagePrefix: "cosign verification failed: failed to verify with keys: failed to process AKV signature: unsupported public key type", + expectedResultMessagePrefix: "Verification failed", + expectedErrorReason: "failed to verify with keys: failed to process AKV signature: unsupported public key type: *ecdh.PublicKey", }, { name: "invalid RSA key size for AKV", @@ -690,7 +697,8 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry blobDigest: validSignatureBlob, }, }, - expectedResultMessagePrefix: "cosign verification failed: failed to verify with keys: failed to process AKV signature: RSA key check: unsupported key size", + expectedResultMessagePrefix: "Verification failed", + expectedErrorReason: "failed to verify with keys: failed to process AKV signature: RSA key check: unsupported key size: 128", }, { name: "invalid ECDSA curve type for AKV", @@ -721,7 +729,8 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry blobDigest: validSignatureBlob, }, }, - expectedResultMessagePrefix: "cosign verification failed: failed to verify with keys: failed to process AKV signature: ECDSA key check: unsupported key curve", + expectedResultMessagePrefix: "Verification failed", + expectedErrorReason: "failed to verify with keys: failed to process AKV signature: ECDSA key check: unsupported key curve: P-224", }, { name: "valid hash: 256 keysize: 2048 RSA key AKV", @@ -761,7 +770,7 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry "sha256:6e1ffef2ba058cda5d1aa7ed792cb1e63b4207d8195a469bee1b5fc662cd9b70": []byte(`{"critical":{"identity":{"docker-reference":"artifactstest.azurecr.io/4-15-24/cosign/hello-world"},"image":{"docker-manifest-digest":"sha256:d37ada95d47ad12224c205a938129df7a3e52345828b4fa27b03a98825d1e2e7"},"type":"cosign container image signature"},"optional":null}`), }, }, - expectedResultMessagePrefix: "cosign verification success", + expectedResultMessagePrefix: "Verification success", }, { name: "valid hash: 256 keysize: 3072 RSA key", @@ -801,7 +810,7 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry "sha256:6e1ffef2ba058cda5d1aa7ed792cb1e63b4207d8195a469bee1b5fc662cd9b70": []byte(`{"critical":{"identity":{"docker-reference":"artifactstest.azurecr.io/4-15-24/cosign/hello-world"},"image":{"docker-manifest-digest":"sha256:d37ada95d47ad12224c205a938129df7a3e52345828b4fa27b03a98825d1e2e7"},"type":"cosign container image signature"},"optional":null}`), }, }, - expectedResultMessagePrefix: "cosign verification success", + expectedResultMessagePrefix: "Verification success", }, { name: "valid hash: 256 curve: P256 ECDSA key AKV", @@ -841,7 +850,7 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry "sha256:6e1ffef2ba058cda5d1aa7ed792cb1e63b4207d8195a469bee1b5fc662cd9b70": []byte(`{"critical":{"identity":{"docker-reference":"artifactstest.azurecr.io/4-15-24/cosign/hello-world"},"image":{"docker-manifest-digest":"sha256:d37ada95d47ad12224c205a938129df7a3e52345828b4fa27b03a98825d1e2e7"},"type":"cosign container image signature"},"optional":null}`), }, }, - expectedResultMessagePrefix: "cosign verification success", + expectedResultMessagePrefix: "Verification success", }, { name: "valid hash: 256 curve: P384 ECDSA key", @@ -881,7 +890,7 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry "sha256:6e1ffef2ba058cda5d1aa7ed792cb1e63b4207d8195a469bee1b5fc662cd9b70": []byte(`{"critical":{"identity":{"docker-reference":"artifactstest.azurecr.io/4-15-24/cosign/hello-world"},"image":{"docker-manifest-digest":"sha256:d37ada95d47ad12224c205a938129df7a3e52345828b4fa27b03a98825d1e2e7"},"type":"cosign container image signature"},"optional":null}`), }, }, - expectedResultMessagePrefix: "cosign verification success", + expectedResultMessagePrefix: "Verification success", }, { name: "successful keyless verification", @@ -917,7 +926,7 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry "sha256:d1226e36bc8502978324cb2cb2116c6aa48edb2ea8f15b1c6f6f256ed43388f6": []byte(`{"critical":{"identity":{"docker-reference":"wabbitnetworks.azurecr.io/test/cosign-image"},"image":{"docker-manifest-digest":"sha256:623621b56649b5e0c2c7cf3ffd987932f8f9a5a01036e00d6f3ae9480087621c"},"type":"cosign container image signature"},"optional":null}`), }, }, - expectedResultMessagePrefix: "cosign verification success", + expectedResultMessagePrefix: "Verification success", }, { name: "failed keyless verification", @@ -953,7 +962,8 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry "sha256:d1226e36bc8502978324cb2cb2116c6aa48edb2ea8f15b1c6f6f256ed43388f6": []byte(`{"critical":{"identity":{"docker-reference":"wabbitnetworks.azurecr.io/test/cosign-image"},"image":{"docker-manifest-digest":"sha256:623621b56649b5e0c2c7cf3ffd987932f8f9a5a01036e00d6f3ae9480087621c"},"type":"cosign container image signature"},"optional":null}`), }, }, - expectedResultMessagePrefix: "cosign verification failed", + expectedResultMessagePrefix: "Verification failed", + expectedErrorReason: "failed to parse static signature opts: failed to unmarshal bundle from blob payload: illegal base64 data at input byte 91", }, } @@ -991,7 +1001,10 @@ mmBwUAwwW0Uc+Nt3bDOCiB1nUsICv1ry } result, _ := cosignVerifier.Verify(context.Background(), subjectRef, refDescriptor, tt.store) if !strings.HasPrefix(result.Message, tt.expectedResultMessagePrefix) { - t.Errorf("Verify() = %v, want %v", result.Message, tt.expectedResultMessagePrefix) + t.Errorf("result.Message = %v, want %v", result.Message, tt.expectedResultMessagePrefix) + } + if result.ErrorReason != tt.expectedErrorReason { + t.Fatalf("expected error reason: %s, but got: %s", tt.expectedErrorReason, result.ErrorReason) } }) } diff --git a/pkg/verifier/notation/notation.go b/pkg/verifier/notation/notation.go index 6fb0ccf0f..1a4349347 100644 --- a/pkg/verifier/notation/notation.go +++ b/pkg/verifier/notation/notation.go @@ -175,11 +175,13 @@ func (v *notationPluginVerifier) Verify(ctx context.Context, } return verifier.VerifierResult{ - Name: v.name, - Type: v.verifierType, - IsSuccess: true, - Message: "signature verification success", - Extensions: extensions, + Name: v.name, + Type: v.verifierType, + VerifierName: v.name, + VerifierType: v.verifierType, + IsSuccess: true, + Message: "Signature verification success", + Extensions: extensions, }, nil } diff --git a/pkg/verifier/types/types.go b/pkg/verifier/types/types.go index 1c12dc086..2eed0a669 100644 --- a/pkg/verifier/types/types.go +++ b/pkg/verifier/types/types.go @@ -47,11 +47,14 @@ const ( // VerifierResult describes the verification result returned from the verifier plugin type VerifierResult struct { - IsSuccess bool `json:"isSuccess"` - Message string `json:"message"` - Name string `json:"name"` - Type string `json:"type,omitempty"` - Extensions interface{} `json:"extensions"` + IsSuccess bool `json:"isSuccess"` + Message string `json:"message"` + ErrorReason string `json:"errorReason,omitempty"` + Name string `json:"name"` + VerifierName string `json:"verifierName,omitempty"` + Type string `json:"type,omitempty"` + VerifierType string `json:"verifierType,omitempty"` + Extensions interface{} `json:"extensions"` } // GetVerifierResult encodes the given JSON data into verify result object @@ -61,11 +64,13 @@ func GetVerifierResult(result []byte) (*verifier.VerifierResult, error) { return nil, err } return &verifier.VerifierResult{ - IsSuccess: vResult.IsSuccess, - Message: vResult.Message, - Name: vResult.Name, - Type: vResult.Type, - Extensions: vResult.Extensions, + IsSuccess: vResult.IsSuccess, + Message: vResult.Message, + Name: vResult.Name, + Type: vResult.Type, + VerifierName: vResult.Name, + VerifierType: vResult.Type, + Extensions: vResult.Extensions, }, nil } @@ -78,9 +83,12 @@ func WriteVerifyResultResult(result *verifier.VerifierResult, w io.Writer) error // verifier.VerifierResult. func NewVerifierResult(result verifier.VerifierResult) VerifierResult { return VerifierResult{ - IsSuccess: result.IsSuccess, - Message: result.Message, - Name: result.Name, - Extensions: result.Extensions, + IsSuccess: result.IsSuccess, + Message: result.Message, + Name: result.Name, + Type: result.Type, + VerifierName: result.VerifierName, + VerifierType: result.VerifierType, + Extensions: result.Extensions, } } From af1a0d8ecc1d7845887118500a25db3de1522704 Mon Sep 17 00:00:00 2001 From: Binbin Li Date: Wed, 31 Jul 2024 08:27:52 +0000 Subject: [PATCH 09/20] feat: refactor error message format --- cmd/ratify/cmd/cmd_test.go | 4 +- errors/types.go | 133 ++++++++++++++++++++++--------------- errors/types_test.go | 127 ++++++++++++++++++++++++----------- test/bats/plugin-test.bats | 2 +- 4 files changed, 173 insertions(+), 93 deletions(-) diff --git a/cmd/ratify/cmd/cmd_test.go b/cmd/ratify/cmd/cmd_test.go index 49149dcf5..9016198ab 100644 --- a/cmd/ratify/cmd/cmd_test.go +++ b/cmd/ratify/cmd/cmd_test.go @@ -50,8 +50,8 @@ func TestDiscover(t *testing.T) { // TODO: make ratify cli more unit testable // unit test should not need to resolve real image - if !strings.Contains(err.Error(), "referrer store failure") { - t.Errorf("error expected") + if !strings.Contains(err.Error(), "REFERRER_STORE_FAILURE") { + t.Errorf("expected containing: %s, but got: %s", "REFERRER_STORE_FAILURE", err.Error()) } } diff --git a/errors/types.go b/errors/types.go index 3a211c80a..a203a8000 100644 --- a/errors/types.go +++ b/errors/types.go @@ -60,15 +60,16 @@ type ErrorCode int // Error provides a wrapper around ErrorCode with extra Details provided. type Error struct { - OriginalError error `json:"originalError,omitempty"` - Code ErrorCode `json:"code"` - Message string `json:"message"` - Detail interface{} `json:"detail,omitempty"` - ComponentType ComponentType `json:"componentType,omitempty"` - PluginName string `json:"pluginName,omitempty"` - LinkToDoc string `json:"linkToDoc,omitempty"` - Stack string `json:"stack,omitempty"` - Description string `json:"description,omitempty"` + originalError error + code ErrorCode + message string + detail interface{} + componentType ComponentType + remediation string + pluginName string + stack string + description string + isRootError bool // isRootError is true if the originalError is either nil or not an Error type } // ErrorDescriptor provides relevant information about a given error code. @@ -152,7 +153,7 @@ func (ec ErrorCode) WithComponentType(componentType ComponentType) Error { // WithLinkToDoc returns a new Error object with attached link to the documentation. func (ec ErrorCode) WithLinkToDoc(link string) Error { - return newError(ec, ec.Message()).WithLinkToDoc(link) + return newError(ec, ec.Message()).WithRemediation(link) } func (ec ErrorCode) WithDescription() Error { @@ -165,27 +166,29 @@ func (ec ErrorCode) WithPluginName(pluginName string) Error { } // NewError returns a new Error object. -func (ec ErrorCode) NewError(componentType ComponentType, pluginName, link string, err error, detail interface{}, printStackTrace bool) Error { +func (ec ErrorCode) NewError(componentType ComponentType, pluginName, remediation string, err error, detail interface{}, printStackTrace bool) Error { stack := "" if printStackTrace { stack = getStackTrace() } return Error{ - Code: ec, - Message: ec.Message(), - OriginalError: err, - Detail: detail, - ComponentType: componentType, - PluginName: pluginName, - LinkToDoc: link, - Stack: stack, + code: ec, + message: ec.Message(), + originalError: err, + detail: detail, + componentType: componentType, + pluginName: pluginName, + remediation: remediation, + stack: stack, + isRootError: err == nil || !errors.As(err, &Error{}), } } func newError(code ErrorCode, message string) Error { return Error{ - Code: code, - Message: message, + code: code, + message: message, + isRootError: true, } } @@ -193,90 +196,114 @@ func newError(code ErrorCode, message string) Error { func (e Error) Is(target error) bool { t := &Error{} if errors.As(target, t) { - return e.Code.ErrorCode() == t.Code.ErrorCode() + return e.code.ErrorCode() == t.code.ErrorCode() } return false } // ErrorCode returns the ID/Value of this Error func (e Error) ErrorCode() ErrorCode { - return e.Code + return e.code } // Unwrap returns the original error func (e Error) Unwrap() error { - return e.OriginalError + return e.originalError } // Error returns a human readable representation of the error. +// An Error message includes the error code, detail from nested errors, root cause and remediation, all separated by ": ". func (e Error) Error() string { - var errStr string - if e.OriginalError != nil { - errStr += fmt.Sprintf("Original Error: (%s), ", e.OriginalError.Error()) + err, details := e.getRootError() + if err.detail != nil { + details = append(details, fmt.Sprintf("%s", err.detail)) } - - errStr += fmt.Sprintf("Error: %s, Code: %s", e.Message, e.Code.String()) - - if e.PluginName != "" { - errStr += fmt.Sprintf(", Plugin Name: %s", e.PluginName) + if err.originalError != nil { + details = append(details, err.originalError.Error()) } - if e.ComponentType != "" { - errStr += fmt.Sprintf(", Component Type: %s", e.ComponentType) + if err.remediation != "" { + details = append(details, err.remediation) } + return fmt.Sprintf("%s: %s", err.ErrorCode().Descriptor().Value, strings.Join(details, ": ")) +} - if e.LinkToDoc != "" { - errStr += fmt.Sprintf(", Documentation: %s", e.LinkToDoc) +// GetDetail returns details from all nested errors. +func (e Error) GetDetail() string { + err, details := e.getRootError() + if err.originalError != nil && err.detail != nil { + details = append(details, fmt.Sprintf("%s", err.detail)) } - if e.Detail != nil { - errStr += fmt.Sprintf(", Detail: %v", e.Detail) - } + return strings.Join(details, ": ") +} - if e.Description != "" { - errStr += fmt.Sprintf(", Description: %v", e.Description) +// GetErrorReason returns the root cause of the error. +func (e Error) GetErrorReason() string { + err, _ := e.getRootError() + if err.originalError != nil { + return err.originalError.Error() } + return fmt.Sprintf("%s", err.detail) +} - if e.Stack != "" { - errStr += fmt.Sprintf(", Stack trace: %s", e.Stack) - } +// GetRemiation returns the remediation of the root error. +func (e Error) GetRemediation() string { + err, _ := e.getRootError() + return err.remediation +} - return errStr +func (e Error) getRootError() (err Error, details []string) { + err = e + for !err.isRootError { + if err.detail != nil { + details = append(details, fmt.Sprintf("%s", err.detail)) + } + var ratifyError Error + if errors.As(err.originalError, &ratifyError) { + err = ratifyError + } else { + // break is unnecessary, but added for safety + break + } + } + return err, details } // WithDetail will return a new Error, based on the current one, but with // some Detail info added func (e Error) WithDetail(detail interface{}) Error { - e.Detail = detail + e.detail = detail return e } // WithPluginName returns a new Error object with pluginName set. func (e Error) WithPluginName(pluginName string) Error { - e.PluginName = pluginName + e.pluginName = pluginName return e } // WithComponentType returns a new Error object with ComponentType set. func (e Error) WithComponentType(componentType ComponentType) Error { - e.ComponentType = componentType + e.componentType = componentType return e } // WithError returns a new Error object with original error. func (e Error) WithError(err error) Error { - e.OriginalError = err + e.originalError = err + e.isRootError = err == nil || !errors.As(err, &Error{}) return e } -// WithLinkToDoc returns a new Error object attached with link to documentation. -func (e Error) WithLinkToDoc(link string) Error { - e.LinkToDoc = link +// WithRemediation returns a new Error object attached with remediation. +func (e Error) WithRemediation(remediation string) Error { + e.remediation = remediation return e } func (e Error) WithDescription() Error { - e.Description = e.Code.Description() + e.description = e.code.Description() return e } diff --git a/errors/types_test.go b/errors/types_test.go index 5e97400d1..0929c4a85 100644 --- a/errors/types_test.go +++ b/errors/types_test.go @@ -22,26 +22,31 @@ import ( ) const ( - testGroup = "test-group" - testValue = "test-value" - testMessage = "test-message" - testDescription = "test-description" - testDetail = "test-detail" - testComponentType = "test-component-type" - testLink = "test-link" - testPluginName = "test-plugin-name" - testErrorString = "Original Error: (Error: , Code: UNKNOWN), Error: test-message, Code: test-value, Plugin Name: test-plugin-name, Component Type: test-component-type, Documentation: test-link, Detail: test-detail" - nonexistentEC = 2000 + testGroup = "test-group" + testErrCode1 = "TEST_ERROR_CODE_1" + testErrCode2 = "TEST_ERROR_CODE_2" + testMessage = "test-message" + testDescription = "test-description" + testDetail1 = "test-detail-1" + testDetail2 = "test-detail-2" + testComponentType1 = "test-component-type-1" + testComponentType2 = "test-component-type-2" + testLink1 = "test-link-1" + testLink2 = "test-link-2" + testPluginName = "test-plugin-name" + nonexistentEC = 2000 ) var ( testEC = Register(testGroup, ErrorDescriptor{ - Value: testValue, + Value: testErrCode1, Message: testMessage, Description: testDescription, }) - testEC2 = Register(testGroup, ErrorDescriptor{}) + testEC2 = Register(testGroup, ErrorDescriptor{ + Value: testErrCode2, + }) ) func TestErrorCode(t *testing.T) { @@ -52,8 +57,9 @@ func TestErrorCode(t *testing.T) { } func TestError(t *testing.T) { - if testEC.Error() != testValue { - t.Fatalf("expected: %s, got: %s", testValue, testEC.Error()) + expectedStr := "test error code 1" + if testEC.Error() != expectedStr { + t.Fatalf("expected: %s, got: %s", expectedStr, testEC.Error()) } } @@ -66,7 +72,7 @@ func TestDescriptor(t *testing.T) { { name: "existing error code", ec: testEC, - expectedValue: testValue, + expectedValue: testErrCode1, }, { name: "nonexistent error code", @@ -92,9 +98,9 @@ func TestMessage(t *testing.T) { } func TestWithDetail(t *testing.T) { - err := testEC.WithDetail(testDetail) - if err.Detail != testDetail { - t.Fatalf("expected detail: %s, got: %s", testDetail, err.Detail) + err := testEC.WithDetail(testDetail1) + if err.detail != testDetail1 { + t.Fatalf("expected detail: %s, got: %s", testDetail1, err.detail) } } @@ -106,35 +112,35 @@ func TestWithError(t *testing.T) { } func TestWithComponentType(t *testing.T) { - err := testEC.WithComponentType(testComponentType) - if err.ComponentType != testComponentType { - t.Fatalf("expected component type: %s, got: %s", testComponentType, err.ComponentType) + err := testEC.WithComponentType(testComponentType1) + if err.componentType != testComponentType1 { + t.Fatalf("expected component type: %s, got: %s", testComponentType1, err.componentType) } } func TestWithLinkToDoc(t *testing.T) { - err := testEC.WithLinkToDoc(testLink) - if err.LinkToDoc != testLink { - t.Fatalf("expected link to doc: %s, got: %s", testLink, err.LinkToDoc) + err := testEC.WithLinkToDoc(testLink1) + if err.remediation != testLink1 { + t.Fatalf("expected link to doc: %s, got: %s", testLink1, err.remediation) } } func TestWithPluginName(t *testing.T) { err := testEC.WithPluginName(testPluginName) - if err.PluginName != testPluginName { - t.Fatalf("expected plugin name: %s, got: %s", testPluginName, err.PluginName) + if err.pluginName != testPluginName { + t.Fatalf("expected plugin name: %s, got: %s", testPluginName, err.pluginName) } } func TestWithDescription(t *testing.T) { err := testEC.WithDescription() - if err.Description != testDescription { - t.Fatalf("expected description: %s, got: %s", testDescription, err.Description) + if err.description != testDescription { + t.Fatalf("expected description: %s, got: %s", testDescription, err.description) } } func TestIs(t *testing.T) { - err := testEC.WithDetail(testDetail) + err := testEC.WithDetail(testDetail1) result := err.Is(err) if !result { t.Fatalf("expected true, got: %v", result) @@ -149,7 +155,7 @@ func TestIs(t *testing.T) { func TestError_ErrorCode(t *testing.T) { err := Error{ - Code: 1, + code: 1, } if err.ErrorCode() != 1 { t.Fatalf("expected 1, got: %d", err.ErrorCode()) @@ -168,17 +174,64 @@ func TestIsEmpty(t *testing.T) { } func TestError_Error(t *testing.T) { - err := testEC.WithPluginName(testPluginName).WithComponentType(testComponentType).WithLinkToDoc(testLink).WithDetail(testDetail).WithError(Error{}).WithDescription() - result := err.Error() - if !strings.HasPrefix(result, testErrorString) { - t.Fatalf("expected string starts with: %s, but got: %s", testErrorString, result) + // Nested errors. + rootErr := testEC.NewError(testComponentType1, "", testLink1, errors.New(testMessage), testDetail1, false) + err := testEC2.WithPluginName(testPluginName).WithComponentType(testComponentType2).WithRemediation(testLink2).WithDetail(testDetail2).WithError(rootErr) + + expectedErrStr := strings.Join([]string{testErrCode1, testDetail2, testDetail1, testMessage, testLink1}, ": ") + if err.Error() != expectedErrStr { + t.Fatalf("expected string: %s, but got: %s", expectedErrStr, err.Error()) + } + + // Single error. + err = testEC.WithDetail(testDetail1) + expectedErrStr = "TEST_ERROR_CODE_1: test-detail-1" + if err.Error() != expectedErrStr { + t.Fatalf("expected string: %s, but got: %s", expectedErrStr, err.Error()) + } +} + +func TestError_GetRootCause(t *testing.T) { + // rootErr contains original error. + rootErr := testEC.NewError(testComponentType1, "", testLink1, errors.New(testMessage), testDetail1, false) + err := testEC.WithPluginName(testPluginName).WithComponentType(testComponentType2).WithRemediation(testLink2).WithDetail(testDetail2).WithError(rootErr) + + if err.GetErrorReason() != testMessage { + t.Fatalf("expected root cause: %v, but got: %v", err.GetErrorReason(), testMessage) + } + + // rootErr does not contain original error. + rootErr = testEC.NewError(testComponentType1, "", testLink1, nil, testDetail1, false) + err = testEC.WithPluginName(testPluginName).WithComponentType(testComponentType2).WithRemediation(testLink2).WithDetail(testDetail2).WithError(rootErr) + + if err.GetErrorReason() != testDetail1 { + t.Fatalf("expected root cause: %v, but got: %v", err.GetErrorReason(), testDetail1) + } +} + +func TestError_GetFullDetails(t *testing.T) { + rootErr := testEC.NewError(testComponentType1, "", testLink1, errors.New(testMessage), testDetail1, false) + err := testEC.WithPluginName(testPluginName).WithComponentType(testComponentType2).WithRemediation(testLink2).WithDetail(testDetail2).WithError(rootErr) + + expectedDetails := strings.Join([]string{testDetail2, testDetail1}, ": ") + if err.GetDetail() != expectedDetails { + t.Fatalf("expected full details: %v, but got: %v", expectedDetails, err.GetDetail()) + } +} + +func TestError_GetRootRemediation(t *testing.T) { + rootErr := testEC.NewError(testComponentType1, "", testLink1, errors.New(testMessage), testDetail1, false) + err := testEC.WithPluginName(testPluginName).WithComponentType(testComponentType2).WithRemediation(testLink2).WithDetail(testDetail2).WithError(rootErr) + + if err.GetRemediation() != testLink1 { + t.Fatalf("expected root remediation: %v, but got: %v", err.GetRemediation(), testLink1) } } func TestNewError(t *testing.T) { - err := testEC.NewError(testComponentType, testPluginName, testLink, Error{}, testDetail, false) + err := testEC.NewError(testComponentType1, testPluginName, testLink1, Error{}, testDetail1, false) - if err.ComponentType != testComponentType || err.PluginName != testPluginName || err.LinkToDoc != testLink || err.Detail != testDetail { - t.Fatalf("expected component type: %s, plugin name: %s, link to doc: %s, detail: %s, but got: %s, %s, %s, %s", testComponentType, testPluginName, testLink, testDetail, err.ComponentType, err.PluginName, err.LinkToDoc, err.Detail) + if err.componentType != testComponentType1 || err.pluginName != testPluginName || err.remediation != testLink1 || err.detail != testDetail1 { + t.Fatalf("expected component type: %s, plugin name: %s, link to doc: %s, detail: %s, but got: %s, %s, %s, %s", testComponentType1, testPluginName, testLink1, testDetail1, err.componentType, err.pluginName, err.remediation, err.detail) } } diff --git a/test/bats/plugin-test.bats b/test/bats/plugin-test.bats index 3d37bbd4e..0cc9dea3a 100644 --- a/test/bats/plugin-test.bats +++ b/test/bats/plugin-test.bats @@ -312,7 +312,7 @@ RATIFY_NAMESPACE=gatekeeper-system sed 's/licensechecker/invalidlicensechecker/' ./config/samples/clustered/verifier/config_v1beta1_verifier_complete_licensechecker.yaml >invalidVerifier.yaml run kubectl apply -f invalidVerifier.yaml assert_success - run bash -c "kubectl describe verifiers.config.ratify.deislabs.io/verifier-license-checker -n ${RATIFY_NAMESPACE} | grep 'Brieferror: Original Error:'" + run bash -c "kubectl describe verifiers.config.ratify.deislabs.io/verifier-license-checker -n ${RATIFY_NAMESPACE} | grep 'Brieferror: PLUGIN_NOT_FOUND:'" assert_success # apply a valid verifier, validate status property shows success From 9eccff67efa4baa5757681d19c05d3c9e8b732ee Mon Sep 17 00:00:00 2001 From: Binbin Li Date: Tue, 6 Aug 2024 03:18:31 +0000 Subject: [PATCH 10/20] chore: remove unused code path --- go.mod | 2 +- pkg/certificateprovider/azurekeyvault/auth.go | 10 +--------- pkg/keymanagementprovider/azurekeyvault/auth.go | 10 +--------- 3 files changed, 3 insertions(+), 19 deletions(-) diff --git a/go.mod b/go.mod index 1c03ba92e..10541c2c2 100644 --- a/go.mod +++ b/go.mod @@ -131,7 +131,7 @@ require ( github.com/Azure/azure-sdk-for-go/sdk/internal v1.8.0 // indirect github.com/Azure/go-autorest v14.2.0+incompatible // indirect github.com/Azure/go-autorest/autorest v0.11.29 - github.com/Azure/go-autorest/autorest/adal v0.9.24 + github.com/Azure/go-autorest/autorest/adal v0.9.24 // indirect github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect github.com/Azure/go-autorest/logger v0.2.1 // indirect github.com/Azure/go-autorest/tracing v0.6.0 // indirect diff --git a/pkg/certificateprovider/azurekeyvault/auth.go b/pkg/certificateprovider/azurekeyvault/auth.go index 4abbbe817..b347000e7 100644 --- a/pkg/certificateprovider/azurekeyvault/auth.go +++ b/pkg/certificateprovider/azurekeyvault/auth.go @@ -28,7 +28,6 @@ import ( "github.com/ratify-project/ratify/pkg/utils/azureauth" "github.com/Azure/go-autorest/autorest" - "github.com/Azure/go-autorest/autorest/adal" ) const ( @@ -37,7 +36,6 @@ const ( // the format for expires_on in UTC without AM/PM expiresOnDateFormat = "1/2/2006 15:04:05 +00:00" - tokenTypeBearer = "Bearer" // For Azure AD Workload Identity, the audience recommended for use is // "api://AzureADTokenExchange" DefaultTokenAudience = "api://AzureADTokenExchange" //nolint @@ -64,13 +62,7 @@ func getAuthorizerForWorkloadIdentity(ctx context.Context, tenantID, clientID, r return nil, fmt.Errorf("failed to acquire token: %w", err) } - token := adal.Token{ - AccessToken: result.AccessToken, - Resource: resource, - Type: tokenTypeBearer, - } - token.ExpiresOn, err = parseExpiresOn(result.ExpiresOn.UTC().Local().Format(expiresOnDateFormat)) - if err != nil { + if _, err = parseExpiresOn(result.ExpiresOn.UTC().Local().Format(expiresOnDateFormat)); err != nil { return nil, fmt.Errorf("failed to parse expires_on: %w", err) } diff --git a/pkg/keymanagementprovider/azurekeyvault/auth.go b/pkg/keymanagementprovider/azurekeyvault/auth.go index d380beb00..cd4d248f7 100644 --- a/pkg/keymanagementprovider/azurekeyvault/auth.go +++ b/pkg/keymanagementprovider/azurekeyvault/auth.go @@ -28,7 +28,6 @@ import ( "github.com/ratify-project/ratify/pkg/utils/azureauth" "github.com/Azure/go-autorest/autorest" - "github.com/Azure/go-autorest/autorest/adal" ) const ( @@ -37,7 +36,6 @@ const ( // the format for expires_on in UTC without AM/PM expiresOnDateFormat = "1/2/2006 15:04:05 +00:00" - tokenTypeBearer = "Bearer" // For Azure AD Workload Identity, the audience recommended for use is // "api://AzureADTokenExchange" DefaultTokenAudience = "api://AzureADTokenExchange" //nolint @@ -64,13 +62,7 @@ func getAuthorizerForWorkloadIdentity(ctx context.Context, tenantID, clientID, r return nil, fmt.Errorf("failed to acquire token: %w", err) } - token := adal.Token{ - AccessToken: result.AccessToken, - Resource: resource, - Type: tokenTypeBearer, - } - token.ExpiresOn, err = parseExpiresOn(result.ExpiresOn.UTC().Local().Format(expiresOnDateFormat)) - if err != nil { + if _, err = parseExpiresOn(result.ExpiresOn.UTC().Local().Format(expiresOnDateFormat)); err != nil { return nil, fmt.Errorf("failed to parse expires_on: %w", err) } From aedb22294d8a2931c954b794b4a908ba0188db37 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 6 Aug 2024 05:11:04 +0000 Subject: [PATCH 11/20] chore: Bump step-security/harden-runner from 2.9.0 to 2.9.1 Bumps [step-security/harden-runner](https://github.com/step-security/harden-runner) from 2.9.0 to 2.9.1. - [Release notes](https://github.com/step-security/harden-runner/releases) - [Commits](https://github.com/step-security/harden-runner/compare/0d381219ddf674d61a7572ddd19d7941e271515c...5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde) --- updated-dependencies: - dependency-name: step-security/harden-runner dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- .github/workflows/build-pr.yml | 2 +- .github/workflows/cache-cleanup.yml | 2 +- .github/workflows/clean-dev-package.yml | 2 +- .github/workflows/codeql.yml | 2 +- .github/workflows/e2e-aks.yml | 2 +- .github/workflows/e2e-cli.yml | 8 ++++---- .github/workflows/e2e-k8s.yml | 2 +- .github/workflows/golangci-lint.yml | 2 +- .github/workflows/high-availability.yml | 2 +- .github/workflows/pr-to-main.yml | 2 +- .github/workflows/publish-charts.yml | 2 +- .github/workflows/publish-cosign-sample.yml | 2 +- .github/workflows/publish-dev-assets.yml | 2 +- .github/workflows/publish-package.yml | 2 +- .github/workflows/publish-sample.yml | 2 +- .github/workflows/quick-start.yml | 2 +- .github/workflows/release.yml | 2 +- .github/workflows/run-full-validation.yml | 2 +- .github/workflows/scan-vulns.yaml | 4 ++-- .github/workflows/scorecards.yml | 2 +- .github/workflows/sync-gh-pages.yml | 2 +- 21 files changed, 25 insertions(+), 25 deletions(-) diff --git a/.github/workflows/build-pr.yml b/.github/workflows/build-pr.yml index bda3e7be5..18ab101d4 100644 --- a/.github/workflows/build-pr.yml +++ b/.github/workflows/build-pr.yml @@ -70,7 +70,7 @@ jobs: environment: azure-test steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/cache-cleanup.yml b/.github/workflows/cache-cleanup.yml index 19fc6777b..5fbb3d0c8 100644 --- a/.github/workflows/cache-cleanup.yml +++ b/.github/workflows/cache-cleanup.yml @@ -12,7 +12,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/clean-dev-package.yml b/.github/workflows/clean-dev-package.yml index da9602c62..98cc5cfff 100644 --- a/.github/workflows/clean-dev-package.yml +++ b/.github/workflows/clean-dev-package.yml @@ -13,7 +13,7 @@ jobs: packages: write steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 8d2f26829..d6a8bfc60 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -27,7 +27,7 @@ jobs: steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/e2e-aks.yml b/.github/workflows/e2e-aks.yml index 604203120..17ed2112a 100644 --- a/.github/workflows/e2e-aks.yml +++ b/.github/workflows/e2e-aks.yml @@ -28,7 +28,7 @@ jobs: contents: read steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/e2e-cli.yml b/.github/workflows/e2e-cli.yml index 87e9980ff..324de264f 100644 --- a/.github/workflows/e2e-cli.yml +++ b/.github/workflows/e2e-cli.yml @@ -14,7 +14,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit @@ -35,7 +35,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit @@ -64,7 +64,7 @@ jobs: contents: read steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit @@ -92,7 +92,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/e2e-k8s.yml b/.github/workflows/e2e-k8s.yml index a8f5ee436..bd03c2dd7 100644 --- a/.github/workflows/e2e-k8s.yml +++ b/.github/workflows/e2e-k8s.yml @@ -26,7 +26,7 @@ jobs: contents: read steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/golangci-lint.yml b/.github/workflows/golangci-lint.yml index cea034b54..dd5bec53a 100644 --- a/.github/workflows/golangci-lint.yml +++ b/.github/workflows/golangci-lint.yml @@ -15,7 +15,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/high-availability.yml b/.github/workflows/high-availability.yml index a7d1b0026..a0cbeaeb1 100644 --- a/.github/workflows/high-availability.yml +++ b/.github/workflows/high-availability.yml @@ -30,7 +30,7 @@ jobs: DAPR_VERSION: ["1.13.2"] steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/pr-to-main.yml b/.github/workflows/pr-to-main.yml index 6b3d2e119..3aecd8b89 100644 --- a/.github/workflows/pr-to-main.yml +++ b/.github/workflows/pr-to-main.yml @@ -13,7 +13,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/publish-charts.yml b/.github/workflows/publish-charts.yml index a9aea641e..cfb578923 100644 --- a/.github/workflows/publish-charts.yml +++ b/.github/workflows/publish-charts.yml @@ -13,7 +13,7 @@ jobs: contents: write steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/publish-cosign-sample.yml b/.github/workflows/publish-cosign-sample.yml index b1957923b..2c5a7d7f6 100644 --- a/.github/workflows/publish-cosign-sample.yml +++ b/.github/workflows/publish-cosign-sample.yml @@ -20,7 +20,7 @@ jobs: id-token: write steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/publish-dev-assets.yml b/.github/workflows/publish-dev-assets.yml index e1334406e..b7b294c53 100644 --- a/.github/workflows/publish-dev-assets.yml +++ b/.github/workflows/publish-dev-assets.yml @@ -17,7 +17,7 @@ jobs: environment: azure-publish steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit - name: Checkout diff --git a/.github/workflows/publish-package.yml b/.github/workflows/publish-package.yml index d191b43d7..32a893d36 100644 --- a/.github/workflows/publish-package.yml +++ b/.github/workflows/publish-package.yml @@ -16,7 +16,7 @@ jobs: contents: read steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit - name: Checkout diff --git a/.github/workflows/publish-sample.yml b/.github/workflows/publish-sample.yml index fc887ca8b..b21e8840e 100644 --- a/.github/workflows/publish-sample.yml +++ b/.github/workflows/publish-sample.yml @@ -19,7 +19,7 @@ jobs: packages: write steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/quick-start.yml b/.github/workflows/quick-start.yml index 46937886e..2ceda4e8e 100644 --- a/.github/workflows/quick-start.yml +++ b/.github/workflows/quick-start.yml @@ -30,7 +30,7 @@ jobs: KUBERNETES_VERSION: ["1.29.2"] steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 7f84c71ca..986eb83c1 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -16,7 +16,7 @@ jobs: contents: write steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/run-full-validation.yml b/.github/workflows/run-full-validation.yml index fa43e9560..a3ad7f595 100644 --- a/.github/workflows/run-full-validation.yml +++ b/.github/workflows/run-full-validation.yml @@ -58,7 +58,7 @@ jobs: environment: azure-test steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/scan-vulns.yaml b/.github/workflows/scan-vulns.yaml index 02601045f..7c0ff23c4 100644 --- a/.github/workflows/scan-vulns.yaml +++ b/.github/workflows/scan-vulns.yaml @@ -23,7 +23,7 @@ jobs: timeout-minutes: 15 steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit @@ -39,7 +39,7 @@ jobs: timeout-minutes: 15 steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml index 9d2d70401..73c211a6b 100644 --- a/.github/workflows/scorecards.yml +++ b/.github/workflows/scorecards.yml @@ -28,7 +28,7 @@ jobs: steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit diff --git a/.github/workflows/sync-gh-pages.yml b/.github/workflows/sync-gh-pages.yml index 28fa3824c..fd44a65d1 100644 --- a/.github/workflows/sync-gh-pages.yml +++ b/.github/workflows/sync-gh-pages.yml @@ -17,7 +17,7 @@ jobs: repository-projects: write steps: - name: Harden Runner - uses: step-security/harden-runner@0d381219ddf674d61a7572ddd19d7941e271515c # v2.9.0 + uses: step-security/harden-runner@5c7944e73c4c2a096b17a9cb74d65b6c2bbafbde # v2.9.1 with: egress-policy: audit From 92ce84f23d513354298453a9bc390ea59ed4ac91 Mon Sep 17 00:00:00 2001 From: Juncheng Zhu <74894646+junczhu@users.noreply.github.com> Date: Tue, 6 Aug 2024 16:27:48 +0800 Subject: [PATCH 12/20] chore: update scorecards action (#1687) --- .github/workflows/scorecards.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml index 73c211a6b..4221034cf 100644 --- a/.github/workflows/scorecards.yml +++ b/.github/workflows/scorecards.yml @@ -8,10 +8,12 @@ on: branches: - main - dev + - release-* pull_request: branches: - dev - main + - release-* workflow_dispatch: permissions: read-all From 51c5402b16b619867c92dcf15789f2d91cbc11aa Mon Sep 17 00:00:00 2001 From: Binbin Li Date: Tue, 6 Aug 2024 13:07:26 +0000 Subject: [PATCH 13/20] chore: rename WithLinkToDoc to WithRemediation --- errors/types.go | 4 ++-- errors/types_test.go | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/errors/types.go b/errors/types.go index a203a8000..cb5e0311b 100644 --- a/errors/types.go +++ b/errors/types.go @@ -151,8 +151,8 @@ func (ec ErrorCode) WithComponentType(componentType ComponentType) Error { return newError(ec, ec.Message()).WithComponentType(componentType) } -// WithLinkToDoc returns a new Error object with attached link to the documentation. -func (ec ErrorCode) WithLinkToDoc(link string) Error { +// WithRemediation returns a new Error object with remediation. +func (ec ErrorCode) WithRemediation(link string) Error { return newError(ec, ec.Message()).WithRemediation(link) } diff --git a/errors/types_test.go b/errors/types_test.go index 0929c4a85..a1f66e388 100644 --- a/errors/types_test.go +++ b/errors/types_test.go @@ -118,10 +118,10 @@ func TestWithComponentType(t *testing.T) { } } -func TestWithLinkToDoc(t *testing.T) { - err := testEC.WithLinkToDoc(testLink1) +func TestWithRemediation(t *testing.T) { + err := testEC.WithRemediation(testLink1) if err.remediation != testLink1 { - t.Fatalf("expected link to doc: %s, got: %s", testLink1, err.remediation) + t.Fatalf("expected remediation: %s, got: %s", testLink1, err.remediation) } } From 46280e0cb4366dccc609ee1ab0d4b3c4b53f5440 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 7 Aug 2024 05:35:57 +0000 Subject: [PATCH 14/20] chore: Bump actions/upload-artifact from 4.3.5 to 4.3.6 Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.5 to 4.3.6. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](https://github.com/actions/upload-artifact/compare/89ef406dd8d7e03cfd12d9e0a4a378f454709029...834a144ee995460fba8ed112a2fc961b36a5ec5a) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- .github/workflows/e2e-aks.yml | 2 +- .github/workflows/e2e-k8s.yml | 2 +- .github/workflows/high-availability.yml | 2 +- .github/workflows/quick-start.yml | 2 +- .github/workflows/release.yml | 2 +- .github/workflows/scorecards.yml | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/workflows/e2e-aks.yml b/.github/workflows/e2e-aks.yml index 17ed2112a..30dced1e8 100644 --- a/.github/workflows/e2e-aks.yml +++ b/.github/workflows/e2e-aks.yml @@ -64,7 +64,7 @@ jobs: make e2e-aks KUBERNETES_VERSION=${{ inputs.k8s_version }} GATEKEEPER_VERSION=${{ inputs.gatekeeper_version }} TENANT_ID=${{ secrets.AZURE_TENANT_ID }} AZURE_SP_OBJECT_ID=${{ secrets.AZURE_SP_OBJECT_ID }} - name: Upload artifacts - uses: actions/upload-artifact@89ef406dd8d7e03cfd12d9e0a4a378f454709029 # v4.3.5 + uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 if: ${{ always() }} with: name: e2e-logs-aks-${{ inputs.k8s_version }}-${{ inputs.gatekeeper_version }} diff --git a/.github/workflows/e2e-k8s.yml b/.github/workflows/e2e-k8s.yml index bd03c2dd7..06d97d5ef 100644 --- a/.github/workflows/e2e-k8s.yml +++ b/.github/workflows/e2e-k8s.yml @@ -65,7 +65,7 @@ jobs: kubectl logs -n gatekeeper-system -l app=ratify --tail=-1 > logs-ratify-preinstall-${{ matrix.KUBERNETES_VERSION }}-${{ matrix.GATEKEEPER_VERSION }}-rego-policy.json kubectl logs -n gatekeeper-system -l app.kubernetes.io/name=ratify --tail=-1 > logs-ratify-${{ matrix.KUBERNETES_VERSION }}-${{ matrix.GATEKEEPER_VERSION }}-rego-policy.json - name: Upload artifacts - uses: actions/upload-artifact@89ef406dd8d7e03cfd12d9e0a4a378f454709029 # v4.3.5 + uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 if: ${{ always() }} with: name: e2e-logs-${{ inputs.k8s_version }}-${{ inputs.gatekeeper_version }} diff --git a/.github/workflows/high-availability.yml b/.github/workflows/high-availability.yml index a0cbeaeb1..09d3fe236 100644 --- a/.github/workflows/high-availability.yml +++ b/.github/workflows/high-availability.yml @@ -60,7 +60,7 @@ jobs: kubectl logs -n gatekeeper-system -l app=ratify --tail=-1 > logs-ratify-preinstall-${{ matrix.DAPR_VERSION }}.json kubectl logs -n gatekeeper-system -l app.kubernetes.io/name=ratify --tail=-1 > logs-ratify-${{ matrix.DAPR_VERSION }}.json - name: Upload artifacts - uses: actions/upload-artifact@89ef406dd8d7e03cfd12d9e0a4a378f454709029 # v4.3.5 + uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 if: ${{ always() }} with: name: e2e-logs-${{ matrix.DAPR_VERSION }} diff --git a/.github/workflows/quick-start.yml b/.github/workflows/quick-start.yml index 2ceda4e8e..ca981e2b5 100644 --- a/.github/workflows/quick-start.yml +++ b/.github/workflows/quick-start.yml @@ -59,7 +59,7 @@ jobs: kubectl logs -n gatekeeper-system -l app=ratify --tail=-1 > logs-ratify-preinstall-${{ matrix.KUBERNETES_VERSION }}-config-policy.json kubectl logs -n gatekeeper-system -l app.kubernetes.io/name=ratify --tail=-1 > logs-ratify-${{ matrix.KUBERNETES_VERSION }}-config-policy.json - name: Upload artifacts - uses: actions/upload-artifact@89ef406dd8d7e03cfd12d9e0a4a378f454709029 # v4.3.5 + uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # v4.3.6 if: ${{ always() }} with: name: e2e-logs-${{ matrix.KUBERNETES_VERSION }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 986eb83c1..28769e410 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -49,7 +49,7 @@ jobs: $RUNNER_TEMP/sbom-tool generate -b . -bc . -pn ratify -pv $GITHUB_REF_NAME -ps Microsoft -nsb https://microsoft.com -V Verbose - name: Upload a Build Artifact - uses: actions/upload-artifact@89ef406dd8d7e03cfd12d9e0a4a378f454709029 # tag=v4.3.5 + uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # tag=v4.3.6 with: name: SBOM SPDX files path: _manifest/spdx_2.2/** diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml index 4221034cf..a35f04db4 100644 --- a/.github/workflows/scorecards.yml +++ b/.github/workflows/scorecards.yml @@ -48,7 +48,7 @@ jobs: publish_results: true - name: "Upload artifact" - uses: actions/upload-artifact@89ef406dd8d7e03cfd12d9e0a4a378f454709029 # tag=v4.3.5 + uses: actions/upload-artifact@834a144ee995460fba8ed112a2fc961b36a5ec5a # tag=v4.3.6 with: name: SARIF file path: results.sarif From f347f6a04d8079353edaa78dc57ad167a228c998 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 7 Aug 2024 05:36:06 +0000 Subject: [PATCH 15/20] chore: Bump github/codeql-action from 3.25.15 to 3.26.0 Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.25.15 to 3.26.0. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/codeql-action/compare/afb54ba388a7dca6ecae48f608c4ff05ff4cc77a...eb055d739abdc2e8de2e5f4ba1a8b246daa779aa) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] --- .github/workflows/codeql.yml | 4 ++-- .github/workflows/scorecards.yml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index d6a8bfc60..c4c55f106 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -38,7 +38,7 @@ jobs: with: go-version: "1.22" - name: Initialize CodeQL - uses: github/codeql-action/init@afb54ba388a7dca6ecae48f608c4ff05ff4cc77a # tag=v3.25.15 + uses: github/codeql-action/init@eb055d739abdc2e8de2e5f4ba1a8b246daa779aa # tag=v3.26.0 with: languages: go - name: Run tidy @@ -46,4 +46,4 @@ jobs: - name: Build CLI run: make build - name: Perform CodeQL Analysis - uses: github/codeql-action/analyze@afb54ba388a7dca6ecae48f608c4ff05ff4cc77a # tag=v3.25.15 + uses: github/codeql-action/analyze@eb055d739abdc2e8de2e5f4ba1a8b246daa779aa # tag=v3.26.0 diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml index 4221034cf..04918ce73 100644 --- a/.github/workflows/scorecards.yml +++ b/.github/workflows/scorecards.yml @@ -55,6 +55,6 @@ jobs: retention-days: 5 - name: "Upload to code-scanning" - uses: github/codeql-action/upload-sarif@afb54ba388a7dca6ecae48f608c4ff05ff4cc77a # tag=v3.25.15 + uses: github/codeql-action/upload-sarif@eb055d739abdc2e8de2e5f4ba1a8b246daa779aa # tag=v3.26.0 with: sarif_file: results.sarif From 742ccc0fc2f0173d49a6e3dff6652e10972a257e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 8 Aug 2024 05:23:02 +0000 Subject: [PATCH 16/20] chore: Bump sigstore/cosign-installer from 3.5.0 to 3.6.0 Bumps [sigstore/cosign-installer](https://github.com/sigstore/cosign-installer) from 3.5.0 to 3.6.0. - [Release notes](https://github.com/sigstore/cosign-installer/releases) - [Commits](https://github.com/sigstore/cosign-installer/compare/59acb6260d9c0ba8f4a2f9d9b48431a222b68e20...4959ce089c160fddf62f7b42464195ba1a56d382) --- updated-dependencies: - dependency-name: sigstore/cosign-installer dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] --- .github/workflows/publish-cosign-sample.yml | 2 +- .github/workflows/publish-dev-assets.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/publish-cosign-sample.yml b/.github/workflows/publish-cosign-sample.yml index 2c5a7d7f6..925284a8c 100644 --- a/.github/workflows/publish-cosign-sample.yml +++ b/.github/workflows/publish-cosign-sample.yml @@ -25,7 +25,7 @@ jobs: egress-policy: audit - name: Install cosign - uses: sigstore/cosign-installer@59acb6260d9c0ba8f4a2f9d9b48431a222b68e20 # v3.5.0 + uses: sigstore/cosign-installer@4959ce089c160fddf62f7b42464195ba1a56d382 # v3.6.0 - name: Get repo run: | diff --git a/.github/workflows/publish-dev-assets.yml b/.github/workflows/publish-dev-assets.yml index b7b294c53..c5dc1ef6a 100644 --- a/.github/workflows/publish-dev-assets.yml +++ b/.github/workflows/publish-dev-assets.yml @@ -25,7 +25,7 @@ jobs: - name: Install Notation uses: notaryproject/notation-action/setup@104aa999103172f827373af8ac14dde7aa6d28f1 # v1.1.0 - name: Install cosign - uses: sigstore/cosign-installer@59acb6260d9c0ba8f4a2f9d9b48431a222b68e20 # v3.5.0 + uses: sigstore/cosign-installer@4959ce089c160fddf62f7b42464195ba1a56d382 # v3.6.0 - name: Az CLI login uses: azure/login@6c251865b4e6290e7b78be643ea2d005bc51f69a # v2.1.1 with: From 23092c6aed2c2172df7942b88a79ac287fb84e1c Mon Sep 17 00:00:00 2001 From: Susan Shi Date: Fri, 9 Aug 2024 16:01:09 +1000 Subject: [PATCH 17/20] feat: add open ssf best practices badge (#1696) Signed-off-by: Susan Shi --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 20daa0dfb..f2b9baafe 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,7 @@ Is a verification engine as a binary executable and on Kubernetes which enables [![Go Report Card](https://goreportcard.com/badge/github.com/ratify-project/ratify)](https://goreportcard.com/report/github.com/ratify-project/ratify) [![build-pr](https://github.com/ratify-project/ratify/actions/workflows/build-pr.yml/badge.svg)](https://github.com/ratify-project/ratify/actions/workflows/build-pr.yml) [![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/ratify-project/ratify/badge)](https://api.securityscorecards.dev/projects/github.com/ratify-project/ratify) +[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/9334/badge)](https://www.bestpractices.dev/projects/9334) [![Go Reference](https://pkg.go.dev/badge/github.com/deislabs/ratify.svg)](https://pkg.go.dev/github.com/deislabs/ratify) ## Table of Contents From 341c545f20aa70c248cc471e83634bb9fabe3f6c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 12 Aug 2024 05:24:57 +0000 Subject: [PATCH 18/20] chore: Bump github.com/google/go-containerregistry from 0.20.1 to 0.20.2 Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry) from 0.20.1 to 0.20.2. - [Release notes](https://github.com/google/go-containerregistry/releases) - [Changelog](https://github.com/google/go-containerregistry/blob/main/.goreleaser.yml) - [Commits](https://github.com/google/go-containerregistry/compare/v0.20.1...v0.20.2) --- updated-dependencies: - dependency-name: github.com/google/go-containerregistry dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- go.mod | 5 ++--- go.sum | 10 ++++------ 2 files changed, 6 insertions(+), 9 deletions(-) diff --git a/go.mod b/go.mod index 10541c2c2..342788269 100644 --- a/go.mod +++ b/go.mod @@ -21,12 +21,12 @@ require ( github.com/dapr/go-sdk v1.8.0 github.com/dgraph-io/ristretto v0.1.1 github.com/distribution/reference v0.5.0 - github.com/docker/cli v26.1.5+incompatible + github.com/docker/cli v27.1.1+incompatible github.com/docker/distribution v2.8.3+incompatible github.com/fsnotify/fsnotify v1.7.0 github.com/go-jose/go-jose/v3 v3.0.3 github.com/golang/protobuf v1.5.4 - github.com/google/go-containerregistry v0.20.1 + github.com/google/go-containerregistry v0.20.2 github.com/gorilla/mux v1.8.1 github.com/notaryproject/notation-core-go v1.0.3 github.com/notaryproject/notation-go v1.0.1 @@ -91,7 +91,6 @@ require ( github.com/digitorus/pkcs7 v0.0.0-20230818184609-3a137a874352 // indirect github.com/digitorus/timestamp v0.0.0-20231217203849-220c5c2851b7 // indirect github.com/dimchansky/utfbom v1.1.1 // indirect - github.com/docker/docker v26.1.5+incompatible // indirect github.com/emicklei/go-restful/v3 v3.11.0 // indirect github.com/evanphx/json-patch v4.12.0+incompatible // indirect github.com/evanphx/json-patch/v5 v5.6.0 // indirect diff --git a/go.sum b/go.sum index 320634ea3..d02ee0209 100644 --- a/go.sum +++ b/go.sum @@ -238,12 +238,10 @@ github.com/dimchansky/utfbom v1.1.1 h1:vV6w1AhK4VMnhBno/TPVCoK9U/LP0PkLCS9tbxHdi github.com/dimchansky/utfbom v1.1.1/go.mod h1:SxdoEBH5qIqFocHMyGOXVAybYJdr71b1Q/j0mACtrfE= github.com/distribution/reference v0.5.0 h1:/FUIFXtfc/x2gpa5/VGfiGLuOIdYa1t65IKK2OFGvA0= github.com/distribution/reference v0.5.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= -github.com/docker/cli v26.1.5+incompatible h1:NxXGSdz2N+Ibdaw330TDO3d/6/f7MvHuiMbuFaIQDTk= -github.com/docker/cli v26.1.5+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= +github.com/docker/cli v27.1.1+incompatible h1:goaZxOqs4QKxznZjjBWKONQci/MywhtRv2oNn0GkeZE= +github.com/docker/cli v27.1.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8= github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk= github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= -github.com/docker/docker v26.1.5+incompatible h1:NEAxTwEjxV6VbBMBoGG3zPqbiJosIApZjxlbrG9q3/g= -github.com/docker/docker v26.1.5+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk= github.com/docker/docker-credential-helpers v0.8.0 h1:YQFtbBQb4VrpoPxhFuzEBPQ9E16qz5SpHLS+uswaCp8= github.com/docker/docker-credential-helpers v0.8.0/go.mod h1:UGFXcuoQ5TxPiB54nHOZ32AWRqQdECoh/Mg0AlEYb40= github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk= @@ -375,8 +373,8 @@ github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/ github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= -github.com/google/go-containerregistry v0.20.1 h1:eTgx9QNYugV4DN5mz4U8hiAGTi1ybXn0TPi4Smd8du0= -github.com/google/go-containerregistry v0.20.1/go.mod h1:YCMFNQeeXeLF+dnhhWkqDItx/JSkH01j1Kis4PsjzFI= +github.com/google/go-containerregistry v0.20.2 h1:B1wPJ1SN/S7pB+ZAimcciVD+r+yV/l/DSArMxlbwseo= +github.com/google/go-containerregistry v0.20.2/go.mod h1:z38EKdKh4h7IP2gSfUUqEvalZBqs6AoLeWfUy34nQC8= github.com/google/go-github/v55 v55.0.0 h1:4pp/1tNMB9X/LuAhs5i0KQAE40NmiR/y6prLNb9x9cg= github.com/google/go-github/v55 v55.0.0/go.mod h1:JLahOTA1DnXzhxEymmFF5PP2tSS9JVNj68mSZNDwskA= github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8= From 2389aa64185236a153cd81be0292d05da511de40 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 12 Aug 2024 05:31:54 +0000 Subject: [PATCH 19/20] chore: Bump vscode/devcontainers/go in /.devcontainer Bumps vscode/devcontainers/go from `dca0f2c` to `8cb4ef6`. --- updated-dependencies: - dependency-name: vscode/devcontainers/go dependency-type: direct:production ... Signed-off-by: dependabot[bot] --- .devcontainer/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile index af5e4d98b..43d46150c 100644 --- a/.devcontainer/Dockerfile +++ b/.devcontainer/Dockerfile @@ -14,7 +14,7 @@ # See here for image contents: https://github.com/microsoft/vscode-dev-containers/tree/v0.245.2/containers/go/.devcontainer/base.Dockerfile # [Choice] Go version (use -bullseye variants on local arm64/Apple Silicon): 1.22-bullseye, 1.21-bullseye, 1, 1.19, 1.18, 1-bullseye, 1.19-bullseye, 1.18-bullseye, 1-buster, 1.19-buster, 1.18-buster -FROM mcr.microsoft.com/vscode/devcontainers/go:1.22-bullseye@sha256:dca0f2ca25750feda8233b2d23c43da5ec1cf1a2afcd04f791981d9e647612ff +FROM mcr.microsoft.com/vscode/devcontainers/go:1.22-bullseye@sha256:8cb4ef68d402b85b52c955ff16410e499cf7604527634d99a2a1922a93751795 # [Choice] Node.js version: none, lts/*, 18, 16, 14 ARG NODE_VERSION="none" From 730c48b9922cf6ed3f83f6776e1538d513299bf9 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 12 Aug 2024 05:48:28 +0000 Subject: [PATCH 20/20] chore: Bump golang from `86a3c48` to `2bd56f0` in /httpserver Bumps golang from `86a3c48` to `2bd56f0`. --- updated-dependencies: - dependency-name: golang dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] --- httpserver/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/httpserver/Dockerfile b/httpserver/Dockerfile index f0bf06d41..48f4caf76 100644 --- a/httpserver/Dockerfile +++ b/httpserver/Dockerfile @@ -11,7 +11,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -FROM --platform=$BUILDPLATFORM golang:1.22@sha256:86a3c48a61915a8c62c0e1d7594730399caa3feb73655dfe96c7bc17710e96cf as builder +FROM --platform=$BUILDPLATFORM golang:1.22@sha256:2bd56f00ff47baf33e64eae7996b65846c7cb5e0a46e0a882ef179fd89654afa as builder ARG TARGETPLATFORM ARG TARGETOS