Official 1.14 Release Docs

content/en/docs/concepts/configuration/pod-priority-preemption.md

@@ -9,7 +9,7 @@ weight: 70
 
 {{% capture overview %}}
 
-{{< feature-state for_k8s_version="1.11" state="beta" >}}
+{{< feature-state for_k8s_version="1.14" state="stable" >}}
 
 [Pods](/docs/user-guide/pods) can have _priority_. Priority indicates the
 importance of a Pod relative to other Pods. If a Pod cannot be scheduled, the
@@ -19,8 +19,8 @@ pending Pod possible.
 In Kubernetes 1.9 and later, Priority also affects scheduling order of Pods and
 out-of-resource eviction ordering on the Node.
 
-Pod priority and preemption are moved to beta since Kubernetes 1.11 and are
-enabled by default in this release and later.
+Pod priority and preemption graduated to beta in Kubernetes 1.11 and to GA in
+Kubernetes 1.14. They have been enabled by default since 1.11.
 
 In Kubernetes versions where Pod priority and preemption is still an alpha-level
 feature, you need to explicitly enable it. To use these features in the older
@@ -34,6 +34,7 @@ Kubernetes Version | Priority and Preemption State | Enabled by default
 1.9                | alpha                         | no
 1.10               | alpha                         | no
 1.11               | beta                          | yes
+1.14               | GA                            | yes
 
 {{< warning >}}In a cluster where not all users are trusted, a
 malicious user could create pods at the highest possible priorities, causing
@@ -71,15 +72,15 @@ Pods.
 ## How to disable preemption
 
 {{< note >}}
-In Kubernetes 1.11, critical pods (except DaemonSet pods, which are
-still scheduled by the DaemonSet controller) rely on scheduler preemption to be
-scheduled when a cluster is under resource pressure. For this reason, you will
-need to run an older version of Rescheduler if you decide to disable preemption.
-More on this is provided below.
+In Kubernetes 1.12+, critical pods rely on scheduler preemption to be scheduled
+when a cluster is under resource pressure. For this reason, it is not
+recommended to disable preemption.
 {{< /note >}}
 
 In Kubernetes 1.11 and later, preemption is controlled by a kube-scheduler flag
 `disablePreemption`, which is set to `false` by default.
+If you want to disable preemption despite the above note, you can set
+`disablePreemption` to `true`.
 
 This option is available in component configs only and is not available in
 old-style command line options. Below is a sample component config to disable
@@ -96,20 +97,6 @@ algorithmSource:
 disablePreemption: true
 ```
 
-### Start an older version of Rescheduler in the cluster
-
-When priority or preemption is disabled, we must run Rescheduler v0.3.1 (instead
-of v0.4.0) to ensure that critical Pods are scheduled when nodes or cluster are
-under resource pressure. Since critical Pod annotation is still supported in
-this release, running Rescheduler should be enough and no other changes to the
-configuration of Pods should be needed.
-
-Rescheduler images can be found at:
-[gcr.io/k8s-image-staging/rescheduler](http://gcr.io/k8s-image-staging/rescheduler).
-
-In the code, changing the Rescheduler version back to v.0.3.1 is the reverse of
-[this PR](https://github.com/kubernetes/kubernetes/pull/65454).
-
 ## PriorityClass
 
 A PriorityClass is a non-namespaced object that defines a mapping from a

content/en/docs/concepts/configuration/scheduler-perf-tuning.md

@@ -8,31 +8,39 @@ weight: 70
 
 {{% capture overview %}}
 
-{{< feature-state for_k8s_version="1.12" >}}
+{{< feature-state for_k8s_version="1.14" state="beta" >}}
 
 Kube-scheduler is the Kubernetes default scheduler. It is responsible for
 placement of Pods on Nodes in a cluster. Nodes in a cluster that meet the
 scheduling requirements of a Pod are called "feasible" Nodes for the Pod. The
 scheduler finds feasible Nodes for a Pod and then runs a set of functions to
 score the feasible Nodes and picks a Node with the highest score among the
-feasible ones to run the Pod. The scheduler then notifies the API server about this
-decision in a process called "Binding".
+feasible ones to run the Pod. The scheduler then notifies the API server about
+this decision in a process called "Binding".
 
 {{% /capture %}}
 
 {{% capture body %}}
 
 ## Percentage of Nodes to Score
 
-Before Kubernetes 1.12, Kube-scheduler used to check the feasibility of all the
-nodes in a cluster and then scored the feasible ones. Kubernetes 1.12 has a new
-feature that allows the scheduler to stop looking for more feasible nodes once
-it finds a certain number of them. This improves the scheduler's performance in
-large clusters. The number is specified as a percentage of the cluster size and
-is controlled by a configuration option called `percentageOfNodesToScore`. The
-range should be between 1 and 100. Other values are considered as 100%. The
-default value of this option is 50%. A cluster administrator can change this value by providing a
-different value in the scheduler configuration. However, it may not be necessary to change this value.
+Before Kubernetes 1.12, Kube-scheduler used to check the feasibility of all
+nodes in a cluster and then scored the feasible ones. Kubernetes 1.12 added a
+new feature that allows the scheduler to stop looking for more feasible nodes
+once it finds a certain number of them. This improves the scheduler's
+performance in large clusters. The number is specified as a percentage of the
+cluster size. The percentage can be controlled by a configuration option called
+`percentageOfNodesToScore`. The range should be between 1 and 100. Larger values
+are considered as 100%. Zero is equivalent to not providing the config option.
+Kubernetes 1.14 has logic to find the percentage of nodes to score based on the
+size of the cluster if it is not specified in the configuration. It uses a
+linear formula which yields 50% for a 100-node cluster. The formula yields 10%
+for a 5000-node cluster. The lower bound for the automatic value is 5%. In other
+words, the scheduler always scores at least 5% of the cluster no matter how
+large the cluster is, unless the user provides the config option with a value 
+smaller than 5.
+
+Below is an example configuration that sets `percentageOfNodesToScore` to 50%.
 
 ```yaml
 apiVersion: componentconfig/v1alpha1
@@ -45,41 +53,37 @@ algorithmSource:
 percentageOfNodesToScore: 50
 ```
 
-{{< note >}}
-In clusters with zero or less than 50 feasible nodes, the
-scheduler still checks all the nodes, simply because there are not enough
-feasible nodes to stop the scheduler's search early.
-{{< /note >}}
+{{< note >}} In clusters with less than 50 feasible nodes, the scheduler still
+checks all the nodes, simply because there are not enough feasible nodes to stop
+the scheduler's search early. {{< /note >}}
 
 **To disable this feature**, you can set `percentageOfNodesToScore` to 100.
 
 ### Tuning percentageOfNodesToScore
 
-`percentageOfNodesToScore` must be a value between 1 and 100
-with the default value of 50. There is also a hardcoded minimum value of 50
-nodes which is applied internally. The scheduler tries to find at
-least 50 nodes regardless of the value of `percentageOfNodesToScore`. This means
-that changing this option to lower values in clusters with several hundred nodes
-will not have much impact on the number of feasible nodes that the scheduler
-tries to find. This is intentional as this option is unlikely to improve
-performance noticeably in smaller clusters. In large clusters with over a 1000
-nodes setting this value to lower numbers may show a noticeable performance
-improvement.
+`percentageOfNodesToScore` must be a value between 1 and 100 with the default
+value being calculated based on the cluster size. There is also a hardcoded
+minimum value of 50 nodes. This means that changing
+this option to lower values in clusters with several hundred nodes will not have
+much impact on the number of feasible nodes that the scheduler tries to find.
+This is intentional as this option is unlikely to improve performance noticeably
+in smaller clusters. In large clusters with over a 1000 nodes setting this value
+to lower numbers may show a noticeable performance improvement.
 
 An important note to consider when setting this value is that when a smaller
 number of nodes in a cluster are checked for feasibility, some nodes are not
 sent to be scored for a given Pod. As a result, a Node which could possibly
 score a higher value for running the given Pod might not even be passed to the
 scoring phase. This would result in a less than ideal placement of the Pod. For
 this reason, the value should not be set to very low percentages. A general rule
-of thumb is to never set the value to anything lower than 30. Lower values
+of thumb is to never set the value to anything lower than 10. Lower values
 should be used only when the scheduler's throughput is critical for your
 application and the score of nodes is not important. In other words, you prefer
 to run the Pod on any Node as long as it is feasible.
 
-It is not recommended to lower this value from its default if your cluster has
-only several hundred Nodes. It is unlikely to improve the scheduler's
-performance significantly.
+If your cluster has several hundred Nodes or fewer, we do not recommend lowering
+the default value of this configuration option. It is unlikely to improve the
+scheduler's performance significantly.
 
 ### How the scheduler iterates over Nodes
 
@@ -91,8 +95,8 @@ for running Pods, the scheduler iterates over the nodes in a round robin
 fashion. You can imagine that Nodes are in an array. The scheduler starts from
 the start of the array and checks feasibility of the nodes until it finds enough
 Nodes as specified by `percentageOfNodesToScore`. For the next Pod, the
-scheduler continues from the point in the Node array that it stopped at when checking
-feasibility of Nodes for the previous Pod.
+scheduler continues from the point in the Node array that it stopped at when
+checking feasibility of Nodes for the previous Pod.
 
 If Nodes are in multiple zones, the scheduler iterates over Nodes in various
 zones to ensure that Nodes from different zones are considered in the

content/en/docs/concepts/services-networking/dns-pod-service.md

@@ -170,10 +170,10 @@ following pod-specific DNS policies. These policies are specified in the
   for details on how DNS queries are handled in those cases.
 - "`ClusterFirstWithHostNet`": For Pods running with hostNetwork, you should
   explicitly set its DNS policy "`ClusterFirstWithHostNet`".
-- "`None`": A new option value introduced in Kubernetes v1.9 (Beta in v1.10). It
-  allows a Pod to ignore DNS settings from the Kubernetes environment. All DNS
-  settings are supposed to be provided using the `dnsConfig` field in the Pod Spec.
-  See [DNS config](#dns-config) subsection below.
+- "`None`": It allows a Pod to ignore DNS settings from the Kubernetes
+  environment. All DNS settings are supposed to be provided using the
+  `dnsConfig` field in the Pod Spec.
+  See [Pod's DNS config](#pod-s-dns-config) subsection below.
 
 {{< note >}}
 "Default" is not the default DNS policy. If `dnsPolicy` is not
@@ -205,13 +205,7 @@ spec:
 
 ### Pod's DNS Config
 
-Kubernetes v1.9 introduces an Alpha feature (Beta in v1.10) that allows users more
-control on the DNS settings for a Pod. This feature is enabled by default in v1.10.
-To enable this feature in v1.9, the cluster administrator
-needs to enable the `CustomPodDNS` feature gate on the apiserver and the kubelet,
-for example, "`--feature-gates=CustomPodDNS=true,...`".
-When the feature gate is enabled, users can set the `dnsPolicy` field of a Pod
-to "`None`" and they can add a new field `dnsConfig` to a Pod Spec.
+Pod's DNS Config allows users more control on the DNS settings for a Pod.
 
 The `dnsConfig` field is optional and it can work with any `dnsPolicy` settings.
 However, when a Pod's `dnsPolicy` is set to "`None`", the `dnsConfig` field has
@@ -257,6 +251,16 @@ search default.svc.cluster.local svc.cluster.local cluster.local
 options ndots:5
 ```
 
+### Feature availability
+
+The availability of Pod DNS Config and DNS Policy "`None`"" is shown as below.
+
+| k8s version | Feature support |
+| :---------: |:-----------:|
+| 1.14 | Stable |
+| 1.10 | Beta (on by default)|
+| 1.9 | Alpha |
+
 {{% /capture %}}
 
 {{% capture whatsnext %}}

content/en/docs/concepts/storage/storage-classes.md

@@ -151,6 +151,11 @@ The following plugins support `WaitForFirstConsumer` with pre-created Persistent
 * All of the above
 * [Local](#local)
 
+{{< feature-state state="beta" for_k8s_version="1.14" >}}
+[CSI volumes](/docs/concepts/storage/volumes/#csi) are also supported with dynamic provisioning
+and pre-created PVs, but you'll need to look at the documentation for a specific CSI driver
+to see its supported topology keys and examples. The `CSINodeInfo` feature gate must be enabled.
+
 ### Allowed Topologies
 
 When a cluster operator specifies the `WaitForFirstConsumer` volume binding mode, it is no longer necessary

content/en/docs/concepts/storage/volumes.md

@@ -1072,13 +1072,14 @@ spec:
 
 ### Using subPath with expanded environment variables
 
-{{< feature-state for_k8s_version="v1.11" state="alpha" >}}
+{{< feature-state for_k8s_version="v1.14" state="alpha" >}}
 
 
-`subPath` directory names can also be constructed from Downward API environment variables.
+Use the `subPathExpr` field to construct `subPath` directory names from Downward API environment variables.
 Before you use this feature, you must enable the `VolumeSubpathEnvExpansion` feature gate.
+The `subPath` and `subPathExpr` properties are mutually exclusive.
 
-In this example, a Pod uses `subPath` to create a directory `pod1` within the hostPath volume `/var/log/pods`, using the pod name from the Downward API.  The host directory `/var/log/pods/pod1` is mounted at `/logs` in the container.
+In this example, a Pod uses `subPathExpr` to create a directory `pod1` within the hostPath volume `/var/log/pods`, using the pod name from the Downward API.  The host directory `/var/log/pods/pod1` is mounted at `/logs` in the container.
 
 ```yaml
 apiVersion: v1
@@ -1099,7 +1100,7 @@ spec:
     volumeMounts:
     - name: workdir1
       mountPath: /logs
-      subPath: $(POD_NAME)
+      subPathExpr: $(POD_NAME)
   restartPolicy: Never
   volumes:
   - name: workdir1
@@ -1216,20 +1217,16 @@ persistent volume:
 
 #### CSI raw block volume support
 
-{{< feature-state for_k8s_version="v1.11" state="alpha" >}}
+{{< feature-state for_k8s_version="v1.14" state="beta" >}}
 
 Starting with version 1.11, CSI introduced support for raw block volumes, which
 relies on the raw block volume feature that was introduced in a previous version of
 Kubernetes.  This feature will make it possible for vendors with external CSI drivers to
 implement raw block volumes support in Kubernetes workloads.
 
-CSI block volume support is feature-gated and turned off by default.  To run CSI with
-block volume support enabled, a cluster administrator must enable the feature for each
-Kubernetes component using the following feature gate flags:
-
-```
---feature-gates=BlockVolume=true,CSIBlockVolume=true
-```
+CSI block volume support is feature-gated, but enabled by default. The two
+feature gates which must be enabled for this feature are `BlockVolume` and
+`CSIBlockVolume`.
 
 Learn how to
 [setup your PV/PVC with raw block volume support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support).

content/en/docs/concepts/workloads/controllers/daemonset.md

@@ -21,7 +21,7 @@ Some typical uses of a DaemonSet are:
 - running a cluster storage daemon, such as `glusterd`, `ceph`, on each node.
 - running a logs collection daemon on every node, such as `fluentd` or `logstash`.
 - running a node monitoring daemon on every node, such as [Prometheus Node Exporter](
-  https://github.com/prometheus/node_exporter), `collectd`, [Dynatrace OneAgent](https://www.dynatrace.com/technologies/kubernetes-monitoring/), [AppDynamics Agent](https://docs.appdynamics.com/display/CLOUD/Container+Visibility+with+Kubernetes), Datadog agent, New Relic agent, Ganglia `gmond` or Instana agent.
+  https://github.com/prometheus/node_exporter), `collectd`, [Dynatrace OneAgent](https://www.dynatrace.com/technologies/kubernetes-monitoring/), [AppDynamics Agent](https://docs.appdynamics.com/display/CLOUD/Container+Visibility+with+Kubernetes), Datadog agent, New Relic agent, Ganglia `gmond`, or [Instana Agent](https://www.instana.com/supported-integrations/kubernetes-monitoring/).
 
 In a simple case, one DaemonSet, covering all nodes, would be used for each type of daemon.
 A more complex setup might use multiple DaemonSets for a single type of daemon, but with

content/en/docs/reference/access-authn-authz/admission-controllers.md

@@ -335,13 +335,6 @@ Examples of information you might put here are:
 
 In any case, the annotations are provided by the user and are not validated by Kubernetes in any way. In the future, if an annotation is determined to be widely useful, it may be promoted to a named field of ImageReviewSpec.
 
-### Initializers (alpha) {#initializers}
-
-The admission controller determines the initializers of a resource based on the existing
-`InitializerConfiguration`s. It sets the pending initializers by modifying the
-metadata of the resource to be created.
-For more information, please check [Dynamic Admission Control](/docs/reference/access-authn-authz/extensible-admission-controllers/).
-
 ### LimitPodHardAntiAffinityTopology {#limitpodhardantiaffinitytopology}
 
 This admission controller denies any pod that defines `AntiAffinity` topology key other than

content/en/docs/reference/access-authn-authz/extensible-admission-controllers.md

@@ -5,6 +5,7 @@ reviewers:
 - whitlockjc
 - caesarxuchao
 - deads2k
+- liggitt
 title: Dynamic Admission Control
 content_template: templates/concept
 weight: 40
@@ -19,16 +20,14 @@ the following:
 * They need to be compiled into kube-apiserver.
 * They are only configurable when the apiserver starts up.
 
-Two features, *Admission Webhooks* (beta in 1.9) and *Initializers* (alpha),
-address these limitations. They allow admission controllers to be developed
-out-of-tree and configured at runtime.
+*Admission Webhooks* (beta in 1.9) addresses these limitations. It allows
+admission controllers to be developed out-of-tree and configured at runtime.
+
+This page describes how to use Admission Webhooks.
 
-This page describes how to use Admission Webhooks and Initializers.
 {{% /capture %}}
 
 {{% capture body %}}
-## Admission Webhooks
-
 ### What are admission webhooks?
 
 Admission webhooks are HTTP callbacks that receive admission requests and do
@@ -196,116 +195,4 @@ users:
 ```
 
 Of course you need to set up the webhook server to handle these authentications.
-
-## Initializers
-
-### What are initializers?
-
-*Initializer* has two meanings:
-
-* A list of pending pre-initialization tasks, stored in every object's metadata
-  (e.g., "AddMyCorporatePolicySidecar").
-
-* A user customized controller, which actually performs those tasks. The name of the task
-  corresponds to the controller which performs the task. For clarity, we call
-  them *initializer controllers* in this page.
-
-Once the controller has performed its assigned task, it removes its name from
-the list. For example, it may send a PATCH that inserts a container in a pod and
-also removes its name from `metadata.initializers.pending`. Initializers may make
-mutations to objects.
-
-Objects which have a non-empty initializer list are considered uninitialized,
-and are not visible in the API unless specifically requested by using the query parameter,
-`?includeUninitialized=true`.
-
-### When to use initializers?
-
-Initializers are useful for admins to force policies (e.g., the
-[AlwaysPullImages](/docs/reference/access-authn-authz/admission-controllers/#alwayspullimages)
-admission controller), or to inject defaults (e.g., the
-[DefaultStorageClass](/docs/reference/access-authn-authz/admission-controllers/#defaultstorageclass)
-admission controller), etc.
-
-{{< note >}}
-If your use case does not involve mutating objects, consider using
-external admission webhooks, as they have better performance.
-{{< /note >}}
-
-### How are initializers triggered?
-
-When an object is POSTed, it is checked against all existing
-`initializerConfiguration` objects (explained below). For all that it matches,
-all `spec.initializers[].name`s are appended to the new object's
-`metadata.initializers.pending` field.
-
-An initializer controller should list and watch for uninitialized objects, by
-using the query parameter `?includeUninitialized=true`. If using client-go, just
-set
-[listOptions.includeUninitialized](https://github.com/kubernetes/kubernetes/blob/v1.13.0/staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go#L332)
-to true.
-
-For the observed uninitialized objects, an initializer controller should first
-check if its name matches `metadata.initializers.pending[0]`. If so, it should then
-perform its assigned task and remove its name from the list.
-
-### Enable initializers alpha feature
-
-*Initializers* is an alpha feature, so it is disabled by default. To turn it on,
-you need to:
-
-* Include "Initializers" in the `--enable-admission-plugins` flag when starting
-  `kube-apiserver`. If you have multiple `kube-apiserver` replicas, all should
-  have the same flag setting.
-
-* Enable the dynamic admission controller registration API by adding
-  `admissionregistration.k8s.io/v1alpha1` to the `--runtime-config` flag passed
-  to `kube-apiserver`, e.g.
-  `--runtime-config=admissionregistration.k8s.io/v1alpha1`. Again, all replicas
-  should have the same flag setting.
-
-### Deploy an initializer controller
-
-You should deploy an initializer controller via the [deployment
-API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#deployment-v1beta1-apps).
-
-### Configure initializers on the fly
-
-You can configure what initializers are enabled and what resources are subject
-to the initializers by creating `initializerConfiguration` resources.
-
-You should first deploy the initializer controller and make sure that it is
-working properly before creating the `initializerConfiguration`. Otherwise, any
-newly created resources will be stuck in an uninitialized state.
-
-The following is an example `initializerConfiguration`:
-
-```yaml
-apiVersion: admissionregistration.k8s.io/v1alpha1
-kind: InitializerConfiguration
-metadata:
-  name: example-config
-initializers:
-  # the name needs to be fully qualified, i.e., containing at least two "."
-  - name: podimage.example.com
-    rules:
-      # apiGroups, apiVersion, resources all support wildcard "*".
-      # "*" cannot be mixed with non-wildcard.
-      - apiGroups:
-          - ""
-        apiVersions:
-          - v1
-        resources:
-          - pods
-```
-
-After you create the `initializerConfiguration`, the system will take a few
-seconds to honor the new configuration. Then, `"podimage.example.com"` will be
-appended to the `metadata.initializers.pending` field of newly created pods. You
-should already have a ready "podimage" initializer controller that handles pods
-whose `metadata.initializers.pending[0].name="podimage.example.com"`. Otherwise
-the pods will be stuck in an uninitialized state.
-
-Make sure that all expansions of the `<apiGroup, apiVersions, resources>` tuple
-in a `rule` are valid. If they are not, separate them in different `rules`.
 {{% /capture %}}

content/en/docs/reference/access-authn-authz/rbac.md

@@ -471,13 +471,18 @@ NOTE: editing the role is not recommended as changes will be overwritten on API
 </tr>
 <tr>
 <td><b>system:basic-user</b></td>
-<td><b>system:authenticated</b> and <b>system:unauthenticated</b> groups</td>
-<td>Allows a user read-only access to basic information about themselves.</td>
+<td><b>system:authenticated</b> group</td>
+<td>Allows a user read-only access to basic information about themselves. Prior to 1.14, this role was also bound to `system:unauthenticated` by default.</td>
 </tr>
 <tr>
 <td><b>system:discovery</b></td>
+<td><b>system:authenticated</b> group</td>
+<td>Allows read-only access to API discovery endpoints needed to discover and negotiate an API level. Prior to 1.14, this role was also bound to `system:unauthenticated` by default.</td>
+</tr>
+<tr>
+<td><b>system:public-info-viewer</b></td>
 <td><b>system:authenticated</b> and <b>system:unauthenticated</b> groups</td>
-<td>Allows read-only access to API discovery endpoints needed to discover and negotiate an API level.</td>
+<td>Allows read-only access to non-sensitive information about the cluster. Introduced in 1.14.</td>
 </tr>
 </table>
 

content/en/docs/reference/command-line-tools-reference/feature-gates.md

@@ -55,9 +55,12 @@ different Kubernetes components.
 | `CPUManager` | `true` | Beta | 1.10 | |
 | `CRIContainerLogRotation` | `false` | Alpha | 1.10 | 1.10 |
 | `CRIContainerLogRotation` | `true` | Beta| 1.11 | |
-| `CSIBlockVolume` | `false` | Alpha | 1.11 | |
-| `CSIDriverRegistry` | `false` | Alpha | 1.12 | |
-| `CSINodeInfo` | `false` | Alpha | 1.12 | |
+| `CSIBlockVolume` | `false` | Alpha | 1.11 | 1.13 |
+| `CSIBlockVolume` | `true` | Beta | 1.14 | |
+| `CSIDriverRegistry` | `false` | Alpha | 1.12 | 1.13 |
+| `CSIDriverRegistry` | `true` | Beta | 1.14 | |
+| `CSINodeInfo` | `false` | Alpha | 1.12 | 1.13 |
+| `CSINodeInfo` | `true` | Beta | 1.14 | |
 | `CSIPersistentVolume` | `false` | Alpha | 1.9 | 1.9 |
 | `CSIPersistentVolume` | `true` | Beta | 1.10 | 1.12 |
 | `CSIPersistentVolume` | `true` | GA | 1.13 | - |
@@ -79,7 +82,8 @@ different Kubernetes components.
 | `DynamicVolumeProvisioning` | `true` | Alpha | 1.3 | 1.7 |
 | `DynamicVolumeProvisioning` | `true` | GA | 1.8 | |
 | `EnableEquivalenceClassCache` | `false` | Alpha | 1.8 | |
-| `ExpandInUsePersistentVolumes` | `false` | Alpha | 1.11 | |
+| `ExpandInUsePersistentVolumes` | `false` | Alpha | 1.11 | 1.13 | |
+| `ExpandInUsePersistentVolumes` | `true` | Beta | 1.14 | |
 | `ExpandPersistentVolumes` | `false` | Alpha | 1.8 | 1.10 |
 | `ExpandPersistentVolumes` | `true` | Beta | 1.11 | |
 | `ExperimentalCriticalPodAnnotation` | `false` | Alpha | 1.5 | |
@@ -142,7 +146,7 @@ different Kubernetes components.
 | `VolumeScheduling` | `false` | Alpha | 1.9 | 1.9 |
 | `VolumeScheduling` | `true` | Beta | 1.10 | 1.12 |
 | `VolumeScheduling` | `true` | GA | 1.13 | |
-| `VolumeSubpathEnvExpansion` | `false` | Alpha | 1.11 | |
+| `VolumeSubpathEnvExpansion` | `false` | Alpha | 1.14 | |
 | `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | - |
 | `ScheduleDaemonSetPods` | `false` | Alpha | 1.11 | 1.11 |
 | `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | |
@@ -309,5 +313,6 @@ Each feature gate is designed for enabling/disabling a specific feature:
   enables the usage of [`local`](/docs/concepts/storage/volumes/#local) volume
   type when used together with the `PersistentLocalVolumes` feature gate.
 - `VolumeSnapshotDataSource`: Enable volume snapshot data source support.
+- `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment variables into a `subPath`.
 
 {{% /capture %}}

content/en/docs/setup/independent/create-cluster-kubeadm.md

@@ -117,6 +117,10 @@ communicates with).
 be passed to kubeadm initialization. Depending on which
 third-party provider you choose, you might need to set the `--pod-network-cidr` to
 a provider-specific value. See [Installing a pod network add-on](#pod-network).
+1. (Optional) Since version 1.14, kubeadm will try to detect the container runtime on Linux
+by using a list of well known domain socket paths. To use different container runtime or
+if there are more than one installed on the provisioned node, specify the `--cri-socket`
+argument to `kubeadm init`. See [Installing runtime](/docs/setup/independent/install-kubeadm/#installing-runtime).
 1. (Optional) Unless otherwise specified, kubeadm uses the network interface associated 
 with the default gateway to advertise the master's IP. To use a different 
 network interface, specify the `--apiserver-advertise-address=<ip-address>` argument 

content/en/docs/setup/independent/install-kubeadm.md

@@ -79,10 +79,28 @@ The pod network plugin you use (see below) may also require certain ports to be
 open. Since this differs with each pod network plugin, please see the
 documentation for the plugins about what port(s) those need.
 
-## Installing runtime
+## Installing runtime {#installing-runtime}
 
 Since v1.6.0, Kubernetes has enabled the use of CRI, Container Runtime Interface, by default.
-The container runtime used by default is Docker, which is enabled through the built-in
+
+Since v1.14.0, kubeadm will try to automatically detect the container runtime on Linux nodes
+by scanning through a list of well known domain sockets. The detectable runtimes and the
+socket paths, that are used, can be found in the table below.
+
+| Runtime    | Domain Socket                    |
+|------------|----------------------------------|
+| Docker     | /var/run/docker.sock             |
+| containerd | /run/containerd/containerd.sock  |
+| CRI-O      | /var/run/crio/crio.sock          |
+
+If both Docker and containerd are detected together, Docker takes precedence. This is
+needed, because Docker 18.09 ships with containerd and both are detectable.
+If any other two or more runtimes are detected, kubeadm will exit with an appropriate
+error message.
+
+On non-Linux nodes the container runtime used by default is Docker.
+
+If the container runtime of choice is Docker, it is used through the built-in
 `dockershim` CRI implementation inside of the `kubelet`.
 
 Other CRI-based runtimes include:

content/en/docs/tasks/administer-cluster/kms-provider.md

@@ -31,7 +31,8 @@ To configure a KMS provider on the API server, include a provider of type ```kms
 
   * `name`: Display name of the KMS plugin.
   * `endpoint`: Listen address of the gRPC server (KMS plugin). The endpoint is a UNIX domain socket.
-  * `cachesize`: Number of data encryption keys (DEKs) to be cached in the clear. When cached, DEKs can be used without another call to the KMS; whereas DEKs that are not cached require a call to the KMS to unwrap.. 
+  * `cachesize`: Number of data encryption keys (DEKs) to be cached in the clear. When cached, DEKs can be used without another call to the KMS; whereas DEKs that are not cached require a call to the KMS to unwrap.
+  * `timeout`: How long should kube-apiserver wait for kms-plugin to respond before returning an error (default is 3 seconds).
 
 See [Understanding the encryption at rest configuration.](/docs/tasks/administer-cluster/encrypt-data)
 
@@ -89,6 +90,7 @@ resources:
         name: myKmsPlugin
         endpoint: unix:///tmp/socketfile.sock
         cachesize: 100
+        timeout: 3s
    - identity: {}
 ```
 

content/en/docs/tasks/extend-kubectl/kubectl-plugins.md

@@ -9,7 +9,7 @@ content_template: templates/task
 
 {{% capture overview %}}
 
-{{< feature-state state="beta" >}}
+{{< feature-state state="stable" >}}
 
 This guide demonstrates how to install and write extensions for [kubectl](/docs/reference/kubectl/kubectl/). By thinking of core `kubectl` commands as essential building blocks for interacting with a Kubernetes cluster, a cluster administrator can think
 of plugins as a means of utilizing these building blocks to create more complex behavior. Plugins extend `kubectl` with new sub-commands, allowing for new and custom features not included in the main distribution of `kubectl`.
@@ -24,8 +24,6 @@ You need to have a working `kubectl` binary installed.
 Plugins were officially introduced as an alpha feature in the v1.8.0 release. They have been re-worked in the v1.12.0 release to support a wider range of use-cases. So, while some parts of the plugins feature were already available in previous versions, a `kubectl` version of 1.12.0 or later is recommended if you are following these docs.
 {{< /note >}}
 
-Until a GA version is released, plugins should be considered unstable, and their underlying mechanism is prone to change.
-
 {{% /capture %}}
 
 {{% capture steps %}}

content/en/docs/user-journeys/users/cluster-operator/intermediate.md

@@ -91,7 +91,7 @@ A common configuration on [Minikube](https://github.com/kubernetes/minikube) and
 There is a [walkthrough of how to install this configuration in your cluster](https://blog.kublr.com/how-to-utilize-the-heapster-influxdb-grafana-stack-in-kubernetes-for-monitoring-pods-4a553f4d36c9).
 As of Kubernetes 1.11, Heapster is deprecated, as per [sig-instrumentation](https://github.com/kubernetes/community/tree/master/sig-instrumentation).  See [Prometheus vs. Heapster vs. Kubernetes Metrics APIs](https://brancz.com/2018/01/05/prometheus-vs-heapster-vs-kubernetes-metrics-apis/) for more information alternatives.
 
-Hosted data analytics services such as [Datadog](https://docs.datadoghq.com/integrations/kubernetes/) also offer Kubernetes integration.
+Hosted monitoring, APM, or data analytics services such as [Datadog](https://docs.datadoghq.com/integrations/kubernetes/) or [Instana](https://www.instana.com/supported-integrations/kubernetes-monitoring/) also offer Kubernetes integration.
 
 ## Additional resources
 

content/zh/docs/tutorials/stateful-application/basic-stateful-set.md

@@ -480,7 +480,7 @@ web-2   k8s.gcr.io/nginx-slim:0.8
 `web-0` has had its image updated, but `web-0` and `web-1` still have the original 
 image. Complete the update by deleting the remaining Pods.
 
-​```shell
+```shell
 kubectl delete pod web-1 web-2
 pod "web-1" deleted
 pod "web-2" deleted
@@ -489,7 +489,7 @@ pod "web-2" deleted
 
 观察 StatefulSet 的 Pod，等待它们全部变成 Running 和 Ready。
 
-```
+```shell
 kubectl get pods -w -l app=nginx
 NAME      READY     STATUS    RESTARTS   AGE
 web-0     1/1       Running   0          8m