diff --git a/content/en/docs/tasks/administer-cluster/cpu-management-policies.md b/content/en/docs/tasks/administer-cluster/cpu-management-policies.md index c1dd6a6f8ccc0..ab800830ea76a 100644 --- a/content/en/docs/tasks/administer-cluster/cpu-management-policies.md +++ b/content/en/docs/tasks/administer-cluster/cpu-management-policies.md @@ -85,7 +85,10 @@ state file `cpu_manager_state` in the kubelet root directory. This policy manages a shared pool of CPUs that initially contains all CPUs in the node. The amount of exclusively allocatable CPUs is equal to the total number of CPUs in the node minus any CPU reservations by the kubelet `--kube-reserved` or -`--system-reserved` options. CPUs reserved by these options are taken, in +`--system-reserved` options. From 1.17, the CPU reservation list can be specified +explictly by kubelet --reserved-cpus option. The explicit CPU list specified by +--reserved-cpus takes precedence over the CPU reservation specified by +--kube-reserved and --system-reserved. CPUs reserved by these options are taken, in integer quantity, from the initial shared pool in ascending order by physical core ID.  This shared pool is the set of CPUs on which any containers in `BestEffort` and `Burstable` pods run. Containers in `Guaranteed` pods with fractional @@ -95,8 +98,8 @@ exclusive CPUs. {{< note >}} The kubelet requires a CPU reservation greater than zero be made -using either `--kube-reserved` and/or `--system-reserved` when the static -policy is enabled. This is because zero CPU reservation would allow the shared +using either `--kube-reserved` and/or `--system-reserved` or `--reserved-cpus` when +the static policy is enabled. This is because zero CPU reservation would allow the shared pool to become empty. {{< /note >}} diff --git a/content/en/docs/tasks/administer-cluster/reserve-compute-resources.md b/content/en/docs/tasks/administer-cluster/reserve-compute-resources.md index c259369bf0309..e7c09b7c9da2b 100644 --- a/content/en/docs/tasks/administer-cluster/reserve-compute-resources.md +++ b/content/en/docs/tasks/administer-cluster/reserve-compute-resources.md @@ -146,6 +146,28 @@ control group (`system.slice` on systemd machines for example). Note that Kubelet **does not** create `--system-reserved-cgroup` if it doesn't exist. Kubelet will fail if an invalid cgroup is specified. +### Explictly Reserved CPU List +{{< feature-state for_k8s_version="v1.17" state="stable" >}} + +- **Kubelet Flag**: `--reserved-cpus=` + +`reserved-cpus` is meant to define an explict CPU set for OS system daemons and +kubernetes system daemons. This option is added in 1.17 release. `reserved-cpus` +is for systems that do not intent to define seperate top level cgroups for +OS system daemons and kubernetes system daemons with regard to cpuset resource. +If the Kubelet **does not** have `--system-reserved-cgroup` and `--kube-reserved-cgroup`, +the explicit cpuset provided by `reserved-cpus` will take precedence over the CPUs +defined by `--kube-reserved` and `--system-reserved` options. + +This option is specifically designed for Telco/NFV use cases where uncontrolled +interrupts/timers may impact the workload performance. you can use this option +to define the explict cpuset for the system/kubernetes daemons as well as the +interrupts/timers, so the rest CPUs on the system can be used exclusively for +workloads, with less impact from uncontrolled interrupts/timers. To move the +system daemon, kubernetes daemons and interrupts/timers to the explict cpuset +defined by this option, other mechanism outside Kubernetes should be used. +For example: in Centos, you can do this using the tuned toolset. + ### Eviction Thresholds - **Kubelet Flag**: `--eviction-hard=[memory.available<500Mi]` @@ -259,4 +281,8 @@ for `kube-reserved` and `system-reserved`. As of Kubernetes version 1.8, the `storage` key name was changed to `ephemeral-storage` for the alpha release. +As of Kubernetes version 1.17, you can optionally specify +explicit cpuset by `reserved-cpus` as CPUs reserved for OS system +daemons/interrupts/timers and Kubernetes daemons. + {{% /capture %}}