Skip to content

Commit

Permalink
update doc for kubelet option --reserved-cpus (#17648)
Browse files Browse the repository at this point in the history
  • Loading branch information
jianzzha authored and Bob Killen committed Dec 6, 2019
1 parent 6fde238 commit fe6087b
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,10 @@ state file `cpu_manager_state` in the kubelet root directory.
This policy manages a shared pool of CPUs that initially contains all CPUs in the
node. The amount of exclusively allocatable CPUs is equal to the total
number of CPUs in the node minus any CPU reservations by the kubelet `--kube-reserved` or
`--system-reserved` options. CPUs reserved by these options are taken, in
`--system-reserved` options. From 1.17, the CPU reservation list can be specified
explictly by kubelet `--reserved-cpus` option. The explicit CPU list specified by
`--reserved-cpus` takes precedence over the CPU reservation specified by
`--kube-reserved` and `--system-reserved`. CPUs reserved by these options are taken, in
integer quantity, from the initial shared pool in ascending order by physical
core ID.  This shared pool is the set of CPUs on which any containers in
`BestEffort` and `Burstable` pods run. Containers in `Guaranteed` pods with fractional
Expand All @@ -95,8 +98,8 @@ exclusive CPUs.

{{< note >}}
The kubelet requires a CPU reservation greater than zero be made
using either `--kube-reserved` and/or `--system-reserved` when the static
policy is enabled. This is because zero CPU reservation would allow the shared
using either `--kube-reserved` and/or `--system-reserved` or `--reserved-cpus` when
the static policy is enabled. This is because zero CPU reservation would allow the shared
pool to become empty.
{{< /note >}}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,28 @@ control group (`system.slice` on systemd machines for example).
Note that Kubelet **does not** create `--system-reserved-cgroup` if it doesn't
exist. Kubelet will fail if an invalid cgroup is specified.

### Explictly Reserved CPU List
{{< feature-state for_k8s_version="v1.17" state="stable" >}}

- **Kubelet Flag**: `--reserved-cpus=0-3`

`reserved-cpus` is meant to define an explict CPU set for OS system daemons and
kubernetes system daemons. This option is added in 1.17 release. `reserved-cpus`
is for systems that do not intent to define seperate top level cgroups for
OS system daemons and kubernetes system daemons with regard to cpuset resource.
If the Kubelet **does not** have `--system-reserved-cgroup` and `--kube-reserved-cgroup`,
the explicit cpuset provided by `reserved-cpus` will take precedence over the CPUs
defined by `--kube-reserved` and `--system-reserved` options.

This option is specifically designed for Telco/NFV use cases where uncontrolled
interrupts/timers may impact the workload performance. you can use this option
to define the explict cpuset for the system/kubernetes daemons as well as the
interrupts/timers, so the rest CPUs on the system can be used exclusively for
workloads, with less impact from uncontrolled interrupts/timers. To move the
system daemon, kubernetes daemons and interrupts/timers to the explict cpuset
defined by this option, other mechanism outside Kubernetes should be used.
For example: in Centos, you can do this using the tuned toolset.

### Eviction Thresholds

- **Kubelet Flag**: `--eviction-hard=[memory.available<500Mi]`
Expand Down Expand Up @@ -259,4 +281,8 @@ for `kube-reserved` and `system-reserved`.
As of Kubernetes version 1.8, the `storage` key name was changed to `ephemeral-storage`
for the alpha release.

As of Kubernetes version 1.17, you can optionally specify
explicit cpuset by `reserved-cpus` as CPUs reserved for OS system
daemons/interrupts/timers and Kubernetes daemons.

{{% /capture %}}

0 comments on commit fe6087b

Please sign in to comment.