-
Notifications
You must be signed in to change notification settings - Fork 475
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add ImagePolicy to support signature verification
This enhancement propose an ImagePolicy CRD to provide a way to set image signature verification configuration on Openshift. Signed-off-by: Qi Wang <[email protected]>
- Loading branch information
Showing
1 changed file
with
336 additions
and
0 deletions.
There are no files selected for viewing
336 changes: 336 additions & 0 deletions
336
enhancements/api-review/add-ImagePolicy-to-config.openshift.io-v1alpha1.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,336 @@ | ||
| --- | ||
| title: add-ImagePolicy-to-config.openshift.io-v1alpha1 | ||
| authors: | ||
| - "@QiWang19" | ||
| reviewers: # Include a comment about what domain expertise a reviewer is expected to bring and what area of the enhancement you expect them to focus on. For example: - "@networkguru, for networking aspects, please look at IP bootstrapping aspect" | ||
| - "@saschagrunert, for API design, implementation details and graduation criteria" | ||
| approvers: # A single approver is preferred, the role of the approver is to raise important questions, help ensure the enhancement receives reviews from all applicable areas/SMEs, and determine when consensus is achieved such that the EP can move forward to implementation. Having multiple approvers makes it difficult to determine who is responsible for the actual approval. | ||
| - "@mrunalp" | ||
| api-approvers: # In case of new or modified APIs or API extensions (CRDs, aggregated apiservers, webhooks, finalizers). If there is no API change, use "None" | ||
| - "@mrunalp" | ||
| creation-date: yyyy-mm-dd | ||
| last-updated: yyyy-mm-dd | ||
| tracking-link: # link to the tracking ticket (for example: Jira Feature or Epic ticket) that corresponds to this enhancement | ||
| - https://issues.redhat.com/browse/OCPNODE-1628 | ||
| see-also: | ||
| - "/enhancements/this-other-neat-thing.md" | ||
| replaces: | ||
| - "/enhancements/that-less-than-great-idea.md" | ||
| superseded-by: | ||
| - "/enhancements/our-past-effort.md" | ||
| --- | ||
|
|
||
| # Add CRD ImagePolicy to config.openshift.io/v1alpha1 | ||
|
|
||
| ## Summary | ||
|
|
||
| This enhancement propose an ImagePolicy CRD to provide a way to set image signature verification configuration on Openshift. | ||
|
|
||
| ## Motivation | ||
|
|
||
| ### User Stories | ||
|
|
||
| As an OpenShift user, I want to verify the container images signed using sigstore | ||
| tools, so that I can utilize the increased security of my software supply chain. | ||
|
|
||
| The user can create an ImagePolicy instance specifying the images/repositories to be verified and their policy. The MCO will write the configuration for signature verification. Once this is done, CRI-O can verify the images/repositories. | ||
|
|
||
| ### Goals | ||
|
|
||
| - MCO container runtime config controller watches ImagePolicy instance in different namespaces and merges the instances for each namespace into a single containers-policy.json in a predefined /path/to/policies/\<NAMESPACE\>.json. | ||
|
|
||
| - CRI-O can verify Cosign signature signed images using configuration from ImagePolicy | ||
|
|
||
| ### Non-Goals | ||
|
|
||
| ImagePolicy CRD does not implements cosign with custom Fulcio and Rekor instance. | ||
|
|
||
| ## Proposal | ||
|
|
||
| ### Workflow Description | ||
|
|
||
| **cluster administrator** is a human user responsible for adding signature verification configurations in | ||
| cluster scope. | ||
| 1. The cluster administrator creates ImagePolicy yaml file with `namespace: opesnhift-config`. | ||
| 2. The cluster administrator creates ImagePolicy instance using `oc create -f imagepolicy.yaml`. | ||
| 3. The cluster administrator can get the merged cluster scope policies from `status` of ImagePolicy instances. | ||
| 4. The cluster administrator can delete the signature verification configuration by deleting its ImagePolicy instances. | ||
|
|
||
| **application administrator** is a human user responsible for | ||
| adding signature verification configurations for specific namespace. | ||
|
|
||
| 1. The application administrator creates ImagePolicy yaml file for namespace that is not `namespace: opesnhift-config` | ||
| 2. The application administrator creates ImagePolicy instance using `oc create -f imagepolicy.yaml`. | ||
| 3. The application administrator can get the merged cluster policy and policy for certain namespace from `status` of ImagePolicy instances. | ||
| 4. The application administrator can delete the signature verification configuration by deleting its ImagePolicy instances. | ||
|
|
||
| ### API Extensions | ||
|
|
||
| #### Type definitions | ||
|
|
||
| ```go | ||
| type ImagePolicySpec struct { | ||
| // images holds images/repositories to be verified | ||
| Images []Image `json:"images"` | ||
| // policy defines the verification policy | ||
| Policy Policy `json:"policy"` | ||
| } | ||
|
|
||
| // Image defines the list of images assigned to a policy. | ||
| // For more information about the format, see the document | ||
| // about the location field: | ||
| // https://github.com/containers/image/blob/main/ | ||
| // docs/containers-policy.json.5.md#docker | ||
| type Image struct { | ||
| Hostname string `json:"hostname,omitempty"` | ||
| } | ||
|
|
||
| type Policy struct { | ||
| // OIDCIssuer contains the expected OIDC issuer. | ||
| // Requires SubjectEmail to be set and KeyData to be empty. | ||
| // Example: "https://expected.OIDC.issuer/" | ||
| OIDCIssuer string `json:"oidcIssuer,omitempty"` | ||
|
|
||
| // SubjectEmail holds the email address of the subject. | ||
| // Example: "[email protected]" | ||
| // Requires OIDCIssuer to be set and KeyData to be empty. | ||
| SubjectEmail string `json:"subjectEmail,omitempty"` | ||
|
|
||
| // KeyData contains inline base64 data of the public key. | ||
| // Can be empty if the image got signed keyless. | ||
| // Requires OIDCIssuer and SubjectEmail to be empty. | ||
| KeyData string `json:"keyData,omitempty"` | ||
|
|
||
| // SignedIdentity specifies what image identity the signature | ||
| // claims about the image. | ||
| SignedIdentity Identity `json:"signedIdentity,omitempty"` | ||
|
|
||
| // FulcioCAData contains inline base64 data for the fulcio | ||
| // CA certificate. | ||
| // Defaults to the base64 encoded contents of: | ||
| // https://raw.githubusercontent.com/sigstore/root-signing/main/ | ||
| // targets/fulcio_v1.crt.pem | ||
| // QUESTION: Can we omit it in favor of the upstream default? | ||
| FulcioCAData string `json:"fulcioCAData,omitempty"` | ||
|
|
||
| // RekorKeyData contains inline base64 data of the rekor public key. | ||
| // Defaults to the base64 encoded contents of: | ||
| // https://raw.githubusercontent.com/sigstore/root-signing/main/ | ||
| // targets/rekor.pub | ||
| // QUESTION: Can we omit it in favor of the upstream default? | ||
| RekorKeyData string `json:"rekorKeyData,omitempty"` | ||
| } | ||
|
|
||
| type Identity struct { | ||
| // Defaults to "matchRepository" | ||
| IdentityMatchPolicy IdentityMatchPolicy `json:"identityMatchPolicy,…"` | ||
| Prefix string `json:"prefix,omitempty"` | ||
| SignedPrefix string `json:"signedPrefix,omitempty"` | ||
| DockerReference string `json:"dockerReference,omitempty"` | ||
| DockerRepository string `json:"dockerRepository,omitempty"` | ||
| } | ||
|
|
||
| type IdentityMatchPolicy string | ||
|
|
||
| const ( | ||
| IdentityMatchPolicyMatchRepository = "matchRepository" | ||
| IdentityMatchPolicyMatchExact = "matchExact" | ||
| IdentityMatchPolicyRemapIdentity = "remapIdentity" | ||
| IdentityMatchPolicyMatchRepoDigestOrExact = "matchRepoDigestOrExact" | ||
| IdentityMatchPolicyExactRepository = "exactRepository" | ||
| IdentityMatchPolicyExactReference = "exactReference" | ||
| ) | ||
|
|
||
| type ImagePolicyStatus struct { | ||
| // PolicyJSON contains the whole policy applied to the namespace | ||
| // which got written to disk. This includes cluster-wide policies | ||
| // from the `openshift-config` namespace as well. | ||
| PolicyJSON string `json:"policyJSON,omitempty"` | ||
|
|
||
| // QUESTION: DO we need conditions? | ||
| Conditions []metav1.Condition `json:"conditions,omitempty"` | ||
| } | ||
| ``` | ||
|
|
||
| ### Implementation Details/Notes/Constraints [optional] | ||
|
|
||
| #### Update container runtime config controller to watch ImagePolicy | ||
|
|
||
| MCO container runtime config controller ImagePolicy syncHandler updates the signature verification configurations: | ||
| - retrieves all the ImagePolicy instances on the cluster. | ||
| - validates conflicting data from ImagePolicy instances. | ||
| - adds following configurations to a single machine config | ||
| - merges the instances for each namespace into a single [containers-policy.json](https://github.com/containers/image/blob/main/docs/containers-policy.json.5.md) in a predefined /path/to/policies/\<NAMESPACE\>.json. | ||
| If no policy exists in a namespace, use the current default of: | ||
|
|
||
| ```yaml | ||
| { "default": [{ "type": "insecureAcceptAnything" }] } | ||
| ``` | ||
|
|
||
| If a single policy exists, use the default: | ||
|
|
||
| ```yaml | ||
| { "default": [{"type": "reject"}] } | ||
| ``` | ||
|
|
||
| - adds [/etc/containers/registries.d/*.yaml](https://github.com/containers/image/blob/main/docs/containers-registries.d.5.md) to allow matching sigstore signatures, for example: | ||
|
|
||
| ```yaml | ||
| docker: | ||
| my-registry/image: | ||
| use-sigstore-attachments: true | ||
| ``` | ||
|
|
||
| Alternatively, we can enable it node wide: | ||
|
|
||
| ```yaml | ||
| default-docker: | ||
| use-sigstore-attachments: true | ||
| ``` | ||
| - merges cluster scoped with namespaced ImagePolicy as a json string and update `status` of each ImagePolicy. | ||
| Once an ImagePolicy object gets created/updated, container runtime config controller will call the syncHandler. | ||
| Once an ImagePolicy object gets deleted, container runtime config controller will delete the machine config and run the syncHandler | ||
|
|
||
| #### Notes | ||
|
|
||
| ImagePolicy is defined as namespaced CRD. `namespace: openshift-config` is used for cluster policy. | ||
|
|
||
| ### Risks and Mitigations | ||
|
|
||
| Risk: The cluster scoped and namespace policies overwrite | ||
|
|
||
| Mitigation: MCO container runtime config controller makes sure namespaced policies for an image cannot override global definitions when merging cluster openshift-config policies and namespaced policies into status field of ImagePolicy instances. | ||
|
|
||
| ### Drawbacks | ||
|
|
||
| ## Design Details | ||
|
|
||
| ### Open Questions [optional] | ||
|
|
||
| ### Test Plan | ||
|
|
||
| **Note:** *Section not required until targeted at a release.* | ||
|
|
||
| MCO container runtime config controller can add unit tests and e2e tests. | ||
| - unit test: verify the policies from ImagePolicy instances merged correctly and the controller writes correct format policy.json | ||
| - e2e test: verify the MCO writes configuration to the correct location | ||
|
|
||
|
|
||
| ### Graduation Criteria | ||
|
|
||
| **Note:** *Section not required until targeted at a release.* | ||
|
|
||
| Define graduation milestones. | ||
|
|
||
| These may be defined in terms of API maturity, or as something else. Initial proposal | ||
| should keep this high-level with a focus on what signals will be looked at to | ||
| determine graduation. | ||
|
|
||
| Consider the following in developing the graduation criteria for this | ||
| enhancement: | ||
|
|
||
| - Maturity levels | ||
| - [`alpha`, `beta`, `stable` in upstream Kubernetes][maturity-levels] | ||
| - `Dev Preview`, `Tech Preview`, `GA` in OpenShift | ||
| - [Deprecation policy][deprecation-policy] | ||
|
|
||
| Clearly define what graduation means by either linking to the [API doc definition](https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-versioning), | ||
| or by redefining what graduation means. | ||
|
|
||
| In general, we try to use the same stages (alpha, beta, GA), regardless how the functionality is accessed. | ||
|
|
||
| [maturity-levels]: https://git.k8s.io/community/contributors/devel/sig-architecture/api_changes.md#alpha-beta-and-stable-versions | ||
| [deprecation-policy]: https://kubernetes.io/docs/reference/using-api/deprecation-policy/ | ||
|
|
||
| **If this is a user facing change requiring new or updated documentation in [openshift-docs](https://github.com/openshift/openshift-docs/), | ||
| please be sure to include in the graduation criteria.** | ||
|
|
||
| **Examples**: These are generalized examples to consider, in addition | ||
| to the aforementioned [maturity levels][maturity-levels]. | ||
|
|
||
| #### Dev Preview -> Tech Preview | ||
|
|
||
| - Ability to utilize the enhancement end to end | ||
| - End user documentation, relative API stability | ||
| - Sufficient test coverage | ||
| - Gather feedback from users rather than just developers | ||
| - Enumerate service level indicators (SLIs), expose SLIs as metrics | ||
| - Write symptoms-based alerts for the component(s) | ||
|
|
||
| #### Tech Preview -> GA | ||
|
|
||
| - More testing (upgrade, downgrade, scale) | ||
| - Sufficient time for feedback | ||
| - Available by default | ||
| - Backhaul SLI telemetry | ||
| - Document SLOs for the component | ||
| - Conduct load testing | ||
| - User facing documentation created in [openshift-docs](https://github.com/openshift/openshift-docs/) | ||
|
|
||
| **For non-optional features moving to GA, the graduation criteria must include | ||
| end to end tests.** | ||
|
|
||
| #### Removing a deprecated feature | ||
|
|
||
| Not applicable. | ||
|
|
||
| ### Upgrade / Downgrade Strategy | ||
|
|
||
| Upgrade expectations: | ||
| In order to use ImagePolicy, an existing cluster will have to make on upgrade. | ||
|
|
||
| Downgrade expectations: | ||
| - Not applicable. | ||
| If `N` does not support ImagePolicy CRD, `N+1`supports IamgePolicy. It is not expected that the ImagePolicy related to the failure `N->N+1`. New resources should not be created during `N->N+1`. | ||
|
|
||
|
|
||
| ### Version Skew Strategy | ||
|
|
||
| The MCO does not require skew check. | ||
|
|
||
| ### Operational Aspects of API Extensions | ||
|
|
||
| - Use-cases that requires over 1000 instances of the CRD is rare, not impacting | ||
| general API throughput. | ||
|
|
||
| #### Failure Modes | ||
|
|
||
| - CR field syntax error failure reported from CLI | ||
| - CR conflicting value failure reported by machine config controller logs | ||
| - MCO rolling out configuration file failure reported by MCO | ||
| - If the signature validation fails, then CRI-O will report that in the same way as for any other pull failure. Further enhancements to the kubelet, CRI and CRI-O error handling can be achieved in future Kubernetes releases. | ||
|
|
||
| The above errors should not impact the overall cluster health | ||
| OCP Node team is likely to be called upon in case of escalation with one of the failure modes. | ||
|
|
||
| #### Support Procedures | ||
|
|
||
| - detect the failure modes in a support situation, describe possible symptoms (events, metrics, | ||
| alerts, which log output in which component) | ||
| - If the ImagePolicy CR does not follow the syntax requirements, api-server will fail when creating the objects. | ||
| - If the ImagePolicy CR is not rolled out by MCO, machine-config-controller logs will shows error with the reason. | ||
|
|
||
| - disable the API extension (e.g. remove MutatingWebhookConfiguration `xyz`, remove APIService `foo`) | ||
| - Disabling/removing the CRD is not possible without removing the CR instances. Customer will lose data. | ||
|
|
||
| - What consequences does it have on existing, running workloads? | ||
| - No impact. CRI-O does not verify signature verification for running containers. | ||
|
|
||
| - What consequences does it have for newly created workloads? | ||
| - In order to verify signature for new pods, ImagePolicy instances should be successfully rolled out before creating new workloads. | ||
|
|
||
| - Does functionality fail gracefully and will work resume when re-enabled without risking | ||
| - User can create another objects if previous objects failed to roll out. | ||
|
|
||
| ## Implementation History | ||
|
|
||
| [OCPNODE-1628: Sigstore Support - OpenShift Container Image Validation (Dev Preview)](https://issues.redhat.com/browse/OCPNODE-1628) epic will keep track of the implementation. | ||
|
|
||
| ## Alternatives | ||
|
|
||
| Not applicable. | ||
|
|
||
| ## Infrastructure Needed [optional] | ||
|
|
||
| - Registry proxies like registry.k8s.io are not natively usable: https://github.com/containers/image/issues/1952 | ||
| Workaround: using remapIdentity |