Skip to content

Commit

Permalink
Migrating from old kep to new template: sig-cluster-lifecycle - kubeadm
Browse files Browse the repository at this point in the history
Migrating from old kep to new template: sig-cluster-lifecycle - kubeadm

Migrating from old kep to new template: sig-cluster-lifecycle - kubeadm

README file from kubeadm to parent folder
  • Loading branch information
Shekharrajak committed Feb 13, 2021
1 parent 5385cde commit f380933
Show file tree
Hide file tree
Showing 19 changed files with 221 additions and 244 deletions.
File renamed without changes.
43 changes: 20 additions & 23 deletions keps/sig-cluster-lifecycle/kubeadm/0023-documentation-for-images.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@


# Documentation for images

Open https://www.websequencediagrams.com/ and paste the spec for the desired image:
Expand All @@ -15,43 +13,43 @@ Open https://www.websequencediagrams.com/ and paste the spec for the desired ima
```
title kubeadm init (interactions with the v1beta1 configuration)
participant "user" as u
participant "user" as u
participant "kubeadm" as k
participant "kubelet" as kk
participant "node\n(api object)" as n
participant "kubeadm-config\nConfigMap" as cm
participant "kubeproxy-config\nConfigMap" as kpcm
participant "kubeproxy-config\nConfigMap" as kpcm
participant "kubelet-config\nConfigMap-1.*" as kcm
u->k:provide\nInitConfiguration (with NodeRegistrationOptions, ControlPlaneConfiguration)\nClusterConfiguration\nkube-proxy component configuration\nkubelet component configuration
u->k:provide\nInitConfiguration (with NodeRegistrationOptions, ControlPlaneConfiguration)\nClusterConfiguration\nkube-proxy component configuration\nkubelet component configuration
k->kk:write kubelet component configuration\nto /var/lib/kubelet/config.yaml
k->kk:write NodeRegistrationOptions\nto /var/lib/kubelet/kubeadm-flags.env
kk->n:start node
k->n:save NodeRegistrationOptions.CRISocket\nto kubeadm.alpha.kubernetes.io/cri-socket annotation
k->n:save NodeRegistrationOptions.CRISocket\nto kubeadm.alpha.kubernetes.io/cri-socket annotation
k->k:use InitConfiguration\n(e.g. tokens)
k->cm:save ClusterConfiguration
k->cm:add Current ControlPlaneConfiguration to ClusterConfiguration.Status
k->kpcm:save kube-proxy component configuration
k->kcm:save kubelet component configuration
k->kpcm:save kube-proxy component configuration
k->kcm:save kubelet component configuration
```

## kubeadm join (and join --control-plane)

```
title kubeadm join and join --control-plane (interactions with the v1beta1 configuration)
participant "user" as u
participant "user" as u
participant "kubeadm" as k
participant "kubeadm-config\nConfigMap" as cm
participant "kubelet-config\nConfigMap-1.*" as kcm
participant "kubelet" as kk
participant "node\n(api object)" as n
u->k:provide\nJoinConfiguration\n(with NodeRegistrationOptions)
u->k:provide\nJoinConfiguration\n(with NodeRegistrationOptions)
k->cm:read ClusterConfiguration
cm->k:
Expand All @@ -62,29 +60,29 @@ kcm->k:
k->kk:write kubelet component configuration\nto /var/lib/kubelet/config.yaml
k->kk:write NodeRegistrationOptions\nto /var/lib/kubelet/kubeadm-flags.env
kk->n:start node
k->n:save NodeRegistrationOptions.CRISocket\nto kubeadm.alpha.kubernetes.io/cri-socket annotation
k->n:save NodeRegistrationOptions.CRISocket\nto kubeadm.alpha.kubernetes.io/cri-socket annotation
k->cm:add new ControlPlaneConfiguration\nto ClusterConfiguration.Status\n(only for join --control-plane)
```

## kubeadm reset
## kubeadm reset

```
title kubeadm reset (interactions with the v1beta1 configuration)
participant "user" as u
participant "user" as u
participant "kubeadm" as k
participant "kubeadm-config\nConfigMap" as cm
participant "node\n(api object)" as n
u->k:
u->k:
k->cm:read ClusterConfiguration
cm->k:
k->cm:remove ControlPlaneConfiguration\nfrom ClusterConfiguration.Status\n(only if the node hosts a control plane instance)
k->n:read kubeadm.alpha.kubernetes.io/cri-socket annotation
k->n:read kubeadm.alpha.kubernetes.io/cri-socket annotation
n->k:
k->k:use CRIsocket\nto delete containers
```
Expand All @@ -94,10 +92,10 @@ k->k:use CRIsocket\nto delete containers
```
title kubeadm upgrade apply (interactions with the v1beta1 configuration)
participant "user" as u
participant "user" as u
participant "kubeadm" as k
participant "kubeadm-config\nConfigMap" as cm
participant "kubeproxy-config\nConfigMap" as kpcm
participant "kubeproxy-config\nConfigMap" as kpcm
participant "kubelet-config\nConfigMap-1.*+1" as kcm
participant "kubelet" as kk
participant "node\n(api object)" as n
Expand All @@ -115,17 +113,17 @@ k->kpcm:read kube-proxy component configuration
kpcm->k:
k->k:update kube-proxy\ncomponent configuration\nusing api machinery
k->kpcm:save updated kube-proxy component configuration
note over kpcm, n:the updated kube-proxy component configuration will\nbe used by the updated kube-proxy DaemonSet
note over kpcm, n:the updated kube-proxy component configuration will\nbe used by the updated kube-proxy DaemonSet
k->kcm:read kubelet component configuration
kcm->k:
k->k:update kubelet\ncomponent configuration\nusing api machinery
k->kcm:save updated kubelet component configuration
k->kcm:save updated kubelet component configuration
k->kk:write kubelet component configuration\nto /var/lib/kubelet/config.yaml
k->kk:write NodeRegistrationOptions\nto /var/lib/kubelet/kubeadm-flags.env
kk->n:start node
note over kcm, n:the updated kubelet component configuration\nwill be used by other nodes\nwhen running\nkubeadm upgrade nodes locally
note over kcm, n:the updated kubelet component configuration\nwill be used by other nodes\nwhen running\nkubeadm upgrade nodes locally
```

Expand All @@ -134,15 +132,14 @@ note over kcm, n:the updated kubelet component configuration\nwill be used by ot
```
title kubeadm upgrade node (interactions with the v1beta1 configuration)
participant "user" as u
participant "user" as u
participant "kubeadm" as k
participant "kubelet-config\nConfigMap-1.*" as kcm
participant "kubelet" as kk
u->k:
u->k:
k->kcm:read kubelet\ncomponent configuration
kcm->k:
k->kk:write kubelet component configuration\nto /var/lib/kubelet/config.yaml
```

Original file line number Diff line number Diff line change
@@ -1,22 +1,3 @@
---
title: Kubeadm Config versioning
authors:
- "@liztio"
owning-sig: sig-cluster-lifecycle
participating-sigs:
reviewers:
- "@timothysc"
approvers:
- TBD
editor: TBD
creation-date: 2018-04-12
last-updated: 2018-04-12
status: provisional
see-also:
replaces:
superseded-by:
---

# Kubeadm Config Versioning

## Table of Contents
Expand All @@ -39,16 +20,16 @@ superseded-by:

## Summary

Kubeadm uses MasterConfiguraton for two distinct but similar operations: Initialising a new cluster and upgrading an existing cluster.
The former is typically created by hand by an administrator.
Kubeadm uses MasterConfiguraton for two distinct but similar operations: Initialising a new cluster and upgrading an existing cluster.
The former is typically created by hand by an administrator.
It is stored on disk and passed to `kubeadm init` via command line flag.
The latter is produced by kubeadm using supplied configuration files, command line options, and internal defaults.
It will be stored in a ConfigMap so upgrade operations can find.
It will be stored in a ConfigMap so upgrade operations can find.

Right now the configuration format is unversioned.
This means configuration file formats can change between kubeadm versions and there's no safe way to update the configuration format.

We propose a stable versioning of this configuration, `v1alpha2` and eventually `v1beta1`.
We propose a stable versioning of this configuration, `v1alpha2` and eventually `v1beta1`.
Version information will be _mandatory_ going forward, both for user-generated configuration files and machine-generated configuration maps.

There as an [existing document][config] describing current Kubernetes best practices around component configuration.
Expand All @@ -57,12 +38,12 @@ There as an [existing document][config] describing current Kubernetes best pract

## Motivation

After 1.10.0, we discovered a bug in the upgrade process.
The `MasterConfiguraton` embedded a [struct that had changed][proxyconfig], which caused a backwards-incompatible change to the configuration format.
After 1.10.0, we discovered a bug in the upgrade process.
The `MasterConfiguraton` embedded a [struct that had changed][proxyconfig], which caused a backwards-incompatible change to the configuration format.
This caused `kubeadm upgrade` to fail, because a newer version of kubeadm was attempting to deserialise an older version of the struct.

Because the configuration is often written and read by different versions of kubeadm compiled by different versions of kubernetes,
it's very important for this configuration file to be well-versioned.
Because the configuration is often written and read by different versions of kubeadm compiled by different versions of kubernetes,
it's very important for this configuration file to be well-versioned.

[proxyconfig]: https://github.com/kubernetes/kubernetes/commit/57071d85ee2c27332390f0983f42f43d89821961

Expand All @@ -87,27 +68,27 @@ The concrete proposal is as follows.
1. Immediately start writing Kind and Version information into the `MasterConfiguraton` struct.
2. Define the previous (1.9) version of the struct as `v1alpha1`.
3. Duplicate the KubeProxyConfig struct that caused the schema change, adding the old version to the `v1alpha1` struct.
3. Create a new `v1alpha2` directory mirroring the existing [`v1alpha1`][v1alpha1], which matches the 1.10 schema.
3. Create a new `v1alpha2` directory mirroring the existing [`v1alpha1`][v1alpha1], which matches the 1.10 schema.
This version need not duplicate the file as well.
2. Warn users if their configuration files do not have a version and kind
4. Use [apimachinery's conversion][conversion] library to design migrations from the old (v1alpha1) versions to the new (v1alpha2) versions
5. Determine the changes for v1beta1
6. With v1beta1, enforce presence of version numbers in config files and ConfigMaps, erroring if not present.

[conversion]: https://godoc.org/k8s.io/apimachinery/pkg/conversion
[v1alpha1]: https://github.com/kubernetes/kubernetes/tree/d7d4381961f4eb2a4b581160707feb55731e324e/cmd/kubeadm/app/apis/kubeadm
[v1alpha1]: https://github.com/kubernetes/kubernetes/tree/d7d4381961f4eb2a4b581160707feb55731e324e/cmd/kubeadm/app/apis/kubeadm

### User Stories [optional]

#### As a user upgrading with Kubeadm, I want the upgrade process to not fail with unfamiliar configuration.

In the past, the haphazard nature of the versioning system has meant it was hard to provide strong guarantees between versions.
Implementing strong version guarantees mean any given configuration generated in the past by kubeadm will work with a future version of kubeadm.
Implementing strong version guarantees mean any given configuration generated in the past by kubeadm will work with a future version of kubeadm.
Deprecations can happen in the future in well-regulated ways.

#### As a infrastructure system using kubeadm, I want to be able to write configuration files that always work.

Having a configuration file that changes without notice makes it very difficult to write software that integrates with kubeadm.
Having a configuration file that changes without notice makes it very difficult to write software that integrates with kubeadm.
By providing strong version guarantees, we can guarantee that the files these tools produce will work with a given version of kubeadm.

### Implementation Details/Notes/Constraints
Expand All @@ -120,7 +101,7 @@ When we do need to interface with the rest of the project, we will do so explici

### Risks and Mitigations

Moving to a strongly versioned configuration from a weakly versioned one must be done carefully so as not break kubeadm for existing users.
Moving to a strongly versioned configuration from a weakly versioned one must be done carefully so as not break kubeadm for existing users.
We can start requiring versions of the existing `v1alpha1` format, issuing warnings to users when Version and Kind aren't present.
These fields can be used today, they're simply ignored.
In the future, we could require them, and transition to using `v1alpha1`.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
title: Kubeadm Config versioning
kep-number: 2498
authors:
- "@liztio"
owning-sig: sig-cluster-lifecycle
participating-sigs:
reviewers:
- "@timothysc"
approvers:
- TBD
editor: TBD
creation-date: 2018-04-12
last-updated: 2018-04-12
status: provisional
see-also:
replaces:
superseded-by:
Original file line number Diff line number Diff line change
@@ -1,29 +1,5 @@
# kubeadm join --control-plane workflow

## Metadata

```yaml
---
title: "kubeadm join --control-plane workflow"
authors:
- "@fabriziopandini"
owning-sig: sig-cluster-lifecycle
reviewers:
- "@chuckha"
- "@detiber"
- "@luxas"
approvers:
- "@luxas"
- "@timothysc"
editor: "@fabriziopandini"
creation-date: 2018-01-28
last-updated: 2019-04-18
status: provisional
see-also:
- KEP 0004
---
```

## Table of Contents

<!-- toc -->
Expand Down Expand Up @@ -298,7 +274,7 @@ The updated join workflow will be the following:
> This decision introduce also a limitation, because HA with local etcd won't be supported when the
> user sets the `--apiserver-advertise-address` of each kube-apiserver instance, including the
> instance on the bootstrap control plane, _equal to the `controlplaneAddress` endpoint_.
2. Apply master taint and label to the node.

3. Update the `kubeadm-config` configMap with the information about the new control plane instance.
Expand Down Expand Up @@ -398,7 +374,7 @@ one control plane instances, but with a new sub-step to be executed on secondary
## Drawbacks

The `kubeadm join --control-plane` workflow requires that some condition are satisfied at `kubeadm init` time,
but the user will be informed about compliance/non compliance of such conditions only when running
but the user will be informed about compliance/non compliance of such conditions only when running
`kubeadm join --control plane`.

## Alternatives
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
title: "kubeadm join --control-plane workflow"
kep-number: 2500
authors:
- "@fabriziopandini"
owning-sig: sig-cluster-lifecycle
reviewers:
- "@chuckha"
- "@detiber"
- "@luxas"
approvers:
- "@luxas"
- "@timothysc"
editor: "@fabriziopandini"
creation-date: 2018-01-28
last-updated: 2019-04-18
status: provisional
see-also:
- KEP 0004
Original file line number Diff line number Diff line change
@@ -1,24 +1,3 @@
---
title: kubeadm phases to beta
authors:
- "@fabriziopandini"
owning-sig: sig-cluster-lifecycle
reviewers:
- "@chuckha"
- "@detiber"
- "@liztio"
- "@neolit123"
approvers:
- "@luxas"
- "@timothysc"
editor: "@fabriziopandini"
creation-date: 2018-03-16
last-updated: 2019-01-22
status: provisional
see-also:
- KEP 0008
---

# kubeadm phases to beta

## Table of Contents
Expand Down Expand Up @@ -72,7 +51,7 @@ duplication of code or in some case inconsistencies between the init and phase i
- This proposal doesn't include any changes of improvements to the actual `kubeadm init`
workflow.
- This proposal doesn't include a plan for implementation of phases in workflows
different than the `kubeadm init` workflow; such plans will be defined by the sig
different than the `kubeadm init` workflow; such plans will be defined by the sig
during release planning for each cycle, and then documented in this KEP.
- v1.14 implementation of phases in the `kubeadm join` workflow
- v1.15 implementation of phases in the `kubeadm upgrade node` workflow
Expand Down Expand Up @@ -104,10 +83,10 @@ own, nested, ordered sequence of phases. For instance:
```bash
preflight Run master pre-flight checks
certs Generates all PKI assets necessary to establish the control plane
/ca Generates a self-signed kubernetes CA
/ca Generates a self-signed kubernetes CA
/apiserver Generates an API server serving certificate and key
...
kubeconfig Generates all kubeconfig files necessary to establish the control plane
kubeconfig Generates all kubeconfig files necessary to establish the control plane
/admin Generates a kubeconfig file for the admin to use and for kubeadm itself
/kubelet Generates a kubeconfig file for the kubelet to use.
...
Expand Down Expand Up @@ -154,7 +133,7 @@ workflows e.g. reuse of phase `certs` in both `kubeadm init` and `kubeadm join`

## Implementation History

* [#61631](https://github.com/kubernetes/kubernetes/pull/61631) First prototype implementation
* [#61631](https://github.com/kubernetes/kubernetes/pull/61631) First prototype implementation
(now outdated)
* v1.13 implementation of phases in the `kubeadm init` workflow
* v1.14 implementation of phases in the `kubeadm join` workflow
Expand All @@ -174,4 +153,4 @@ approach.

It is possible to graduate phases by simply moving corresponding command to first level,
but this approach provide less opportunities for reducing the effort
for maintaining phases up to date with the changes of kubeadm.
for maintaining phases up to date with the changes of kubeadm.
Loading

0 comments on commit f380933

Please sign in to comment.