Enabling seccomp profile for pod lead to higher CPU

[xiangyanw]

What happened:
In the worker node running AL2 EKS AMI amazon-eks-node-1.24-v20230825 (containerd 1.6.19, runc 1.1.7, libseccomp 2.4.1), if I enable RuntimeDefault seccomp profile for a pod, it will consume higher CPU.

For example, if I install redis without seccomp profile, each replica takes minor 10m CPU when it's in empty load.

helm upgrade -i -n default redis oci://registry-1.docker.io/bitnamicharts/redis --set global.storageClass=ebs-sc --set replica.containerSecurityContext.seccompProfile=null

$ kubectl top po
NAME               CPU(cores)   MEMORY(bytes)   
redis-master-0     89m          2Mi             
redis-replicas-0   4m           2Mi             
redis-replicas-1   7m           2Mi             
redis-replicas-2   9m           2Mi

If I install redis with seccomp profile, each replica takes 10 times (about 100m) CPU when it's in empty load.

helm upgrade -i -n default redis oci://registry-1.docker.io/bitnamicharts/redis --set global.storageClass=ebs-sc

$ kubectl top po
NAME               CPU(cores)   MEMORY(bytes)   
redis-master-0     102m         2Mi             
redis-replicas-0   99m          2Mi             
redis-replicas-1   92m          2Mi             
redis-replicas-2   116m         2Mi

What you expected to happen:
CPU usage is the same even after enabling seccomp for pod.

How to reproduce it (as minimally and precisely as possible):
Steps as above.

Anything else we need to know?:
This is because of a known issue #153 in libseccomp.

It has been fixed in PR #156.

However, this fix is not in our libseccomp version 2.4.1.

Environment:

• AWS Region: ap-northeast-1
• Instance Type(s): c5.xlarge
• EKS Platform version (use aws eks describe-cluster --name <name> --query cluster.platformVersion): eks.10
• Kubernetes version (use aws eks describe-cluster --name <name> --query cluster.version): 1.24
• AMI Version: 1.24-v20230825
• Kernel (e.g. uname -a): Linux ip-172-31-72-112.ap-northeast-1.compute.internal 5.10.186-179.751.amzn2.x86_64 Template is missing source_ami_id in the variables section #1 SMP Tue Aug 1 20:51:38 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
• Release information (run cat /etc/eks/release on a node):

BASE_AMI_ID="ami-0f2b325398f933a81"
BUILD_TIME="Fri Aug 25 20:04:37 UTC 2023"
BUILD_KERNEL="5.10.186-179.751.amzn2.x86_64"
ARCH="x86_64"

[RichardLeeY]

I also have same issue with bitinami/redis version 17.14.0 or higher. Because from version 17.14.0, These chart have seccompProfile.type set with RunttimeDefault.
I try to replace usr/sbin/runc on eks worker node with runc 1.1.7's latest binary version from here.
The cpu usage down from 80m to 7m.
Hope this helps.

[cartermckinnon]

We're on runc 1.7.x, please let me know if this is still an issue.