Enable warning/complain-mode instead of rejection

[JAORMX]

Description

ClusterImagePolicy objects already provide a lot of flexibility in terms of allowing images from certain organizations and configuring extensively the authorities to trust for others. While the current supported scenarios are a great end-state, the transition period becomes a little complicated. e.g. an organization that's starting to adopt signed images might want to do so gradually.

In order to promote signed images while still allowing folks to deploy, the proposal is to enable a warning mode or complain mode.

The mode would be an attribute of the spec, and could contain the values enforce or warn. Any authorities specified in the spec would still be evaluated, but validation wouldn't fail if the signatures are missing.

A relevant CIP would look as follows:

apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
  name: metal-toolbox-signed
spec:
  mode: warn
  images:
    - glob: "ghcr.io/metal-toolbox/**"
  authorities:
    - keyless:
        url: https://fulcio.sigstore.dev

Warnings would be issued using Kubernetes' warning feature [1], where we can pass a list of warnings to the AdmissionReview response, which will in turn be interpreted and exposed to the caller.

Support for warnings is already present in the Knative pkg module which is used extensively in policy-controller [2]. The current implementation relies on the field error abstraction which is already used by the policy webhook, so it's a matter of setting the appropriate level to make this work.

Further iterations could also take into use audit annotations [3], but support for this would be required in the Knative libraries first.

Finally, in order to give operators further visibility on the progress of signed container image adoption, we could introduce metrics that output the number of warnings that the webhook has issued.

[1] https://kubernetes.io/blog/2020/09/03/warnings/#admission-webhooks
[2] https://github.com/knative/pkg/blob/main/apis/field_error.go#L43
[3] https://kubernetes.io/blog/2020/09/03/warnings/#audit-annotations

[hectorj2f]

In order to promote signed images while still allowing folks to deploy, the proposal is to enable a warning mode or complain mode.

@JAORMX I remember the team discussed and considered this option in the past. I am not sure about all the benefits it could bring to the controller and the conclusion of our discussion then.
Generally I would argue warning messages are generally ignored. I'd suggest to avoid creating any cluster image policy (in a first place) or either remove the namespace annotation until the signed images have been rollout.
Even if you add a mode warn to the policy, I am not sure if the removal of that field in the policy definition would trigger any desired action to running unsigned images.

[JAORMX]

@hectorj2f I agree, errors can be easily ignored. This is why I proposed metrics support as well, this allows a centralized platform team to track progress and display the containers with warnings in a dashboard. Eventually folks can adopt it to their goals to fix the warnings and eventually tighten the policy.