Seccomp profile in queue-proxy incompatible with gvisor

[daraghlowe]

What version of Knative?

1.8.0

Expected Behavior

Pods should be able to start on GKE nodes running gvisor.

Actual Behavior

Gvisor refuses to allow the pods to start as a seccomp profile has been set, the following error is shown in events which refuses to allow the pod to start:

Seccomp is not supported

#13376 added the config below to queue-proxy containers by default, however gvisor won't allow any profile or even a blank profile to be set.

          seccompProfile:
            type: RuntimeDefault

Maybe this could be configurable in a config map whether it gets added to the queue proxy?

Steps to Reproduce the Problem

Create a Knative service running on node running gvisor on a cluster running Knative 1.8.0.

[dprotaso]

cc @skonto @evankanderson

[dprotaso]

@daraghlowe I'm assuming you're dropping the seccompProfile from the installation yamls correct?

[dprotaso]

I'm sorta inclined to roll this change back. Operators can enforce a RuntimeDefault seccomp for all workloads

https://kubernetes.io/docs/tutorials/security/seccomp/#enable-the-use-of-runtimedefault-as-the-default-seccomp-profile-for-all-workloads

[dprotaso]

/triage accepted

[daraghlowe]

I'm assuming you're dropping the seccompProfile from the installation yamls correct?

We run the Knative control plane on a non gvisor node pool so we didn't have any issues upgrading. Our Knative services run on a gvisor node pool however so we only saw the problem with the queue-proxy container.

[skonto]

@dprotaso
My proposal would be:
a) rollback since it breaks gvisor which does not allow any profile, so upgrades are unblocked for services.
b) make this configurable (if not already) because people may want to add the profile only for Knative and not for all workloads.

[dprotaso]

I created a PR to address a)

I think b) would be covered by Evan's PR #13398

[dprotaso]

I wonder if we should remove the default profile in our control plane release yamls as well.

If someone is running the Knative control plane in gVisor or an earlier OpenShift release it will fail to upgrade.

I think the sensible thing is to wait till our k8s min version supports 1.25 to introduce these defaults. I'm assuming that's when they have an effect.

Though in the gVisor situation it'll never work so maybe we encourage folks adopt the operator that can toggle this on and off. Or they do their own post processing of our yamls.

[daraghlowe]

Thanks for the fix @dprotaso

[nak3]

EDIT: The comment is just same with Dave's #13471 (comment) 😅 I think release yaml should have the consistency.

[dprotaso]

Created a new issue

[dprotaso]

From: #13512 (comment)

Seeing how prolific this setting is (many Knative repos and even their dependencies) it's not worth the hassle. Even if we make the changes end-users will have to customize yamls (ie. cert-manager) for this to work.

[kevinGC]

@daraghlowe can you clarify what version of GKE you're using?

[daraghlowe]

@daraghlowe can you clarify what version of GKE you're using?

@kevinGC We're using 1.24.4-gke.800 at the moment in the project we ran into this issue.

[kevinGC]

Looks like knative rolled this back, but anyways per the GKE bug tracker RuntimeDefault and Unconfined are allowed in GKE Sandbox starting in 1.26.

[daraghlowe]

Great stuff, thanks @kevinGC