diff --git a/keps/prod-readiness/sig-network/2681.yaml b/keps/prod-readiness/sig-network/2681.yaml new file mode 100644 index 00000000000..0dd2a67eaaf --- /dev/null +++ b/keps/prod-readiness/sig-network/2681.yaml @@ -0,0 +1,3 @@ +kep-number: 2681 +alpha: + approver: "@wojtek-t" diff --git a/keps/sig-network/2681-pod-host-ip/README.md b/keps/sig-network/2681-pod-host-ip/README.md new file mode 100644 index 00000000000..88cac8a4165 --- /dev/null +++ b/keps/sig-network/2681-pod-host-ip/README.md @@ -0,0 +1,359 @@ +# KEP-2681: Field status.hostIPs added for Pod + + +- [Release Signoff Checklist](#release-signoff-checklist) +- [Summary](#summary) +- [Motivation](#motivation) + - [Goals](#goals) + - [Non-Goals](#non-goals) +- [Proposal](#proposal) + - [User Stories](#user-stories) + - [Story 1](#story-1) + - [Risks and Mitigations](#risks-and-mitigations) +- [Design Details](#design-details) + - [Versioned API Change: PodStatus v1 core](#versioned-api-change-podstatus-v1-core) + - [PodStatus Internal Representation](#podstatus-internal-representation) + - [Maintaining Compatible Interworking between Old and New Clients](#maintaining-compatible-interworking-between-old-and-new-clients) + - [Container Environment Variables](#container-environment-variables) + - [Test Plan](#test-plan) + - [Expected behavior](#expected-behavior) + - [Graduation Criteria](#graduation-criteria) + - [Alpha](#alpha) + - [Alpha -> Beta Graduation](#alpha---beta-graduation) + - [Beta -> GA Graduation](#beta---ga-graduation) + - [Removing a Deprecated Flag](#removing-a-deprecated-flag) + - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) + - [Version Skew Strategy](#version-skew-strategy) +- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire) + - [Feature Enablement and Rollback](#feature-enablement-and-rollback) + - [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning) + - [Monitoring Requirements](#monitoring-requirements) + - [Dependencies](#dependencies) + - [Scalability](#scalability) + - [Troubleshooting](#troubleshooting) +- [Implementation History](#implementation-history) +- [Drawbacks](#drawbacks) +- [Alternatives](#alternatives) +- [Infrastructure Needed (Optional)](#infrastructure-needed-optional) + + +## Release Signoff Checklist + +Items marked with (R) are required *prior to targeting to a milestone / release*. + +- [x] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR) +- [ ] (R) KEP approvers have approved the KEP status as `implementable` +- [ ] (R) Design details are appropriately documented +- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors) + - [ ] e2e Tests for all Beta API Operations (endpoints) + - [ ] (R) Ensure GA e2e tests for meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) + - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free +- [ ] (R) Graduation criteria is in place + - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) +- [ ] (R) Production readiness review completed +- [ ] (R) Production readiness review approved +- [ ] "Implementation History" section is up-to-date for milestone +- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io] +- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes + +[kubernetes.io]: https://kubernetes.io/ +[kubernetes/enhancements]: https://git.k8s.io/enhancements +[kubernetes/kubernetes]: https://git.k8s.io/kubernetes +[kubernetes/website]: https://git.k8s.io/website + +## Summary + +The proposal aims to improve the Pod's ability to obtain the address of the node + +## Motivation + +### Goals + +- Field `status.hostIPs` added for Pod +- Downward API support for `status.hostIPs` + +### Non-Goals + +## Proposal + +### User Stories + +#### Story 1 + +Applications that originally used IPv4 migrated to IPv6 during the dual-stack transition phase, +For smooth migration, IP-related attributes should have both IPv4 and IPv6. + +### Risks and Mitigations + + + +## Design Details + +### Versioned API Change: PodStatus v1 core + +In order to maintain backwards compatibility for the core V1 API, this proposal +retains the existing (singular) "HostIP" field in the core V1 version of the +PodStatus V1 core API and adds a new array of structures that store host IPs +along with associated metadata for that IP. + +``` go + // HostIP represents the IP address of a host. + // IP address information. Each entry includes: + // IP: An IP address allocated to the host. + type HostIP struct { + // ip is an IP address (IPv4 or IPv6) assigned to the host + IP string + } + + // HostIPs holds the IP addresses allocated to the host. If this field is specified, the 0th entry must + // match the hostIP field. This list is empty if no IPs have been allocated yet. + // +optional + HostIPs []HostIP +``` + +### PodStatus Internal Representation + +PodStatus internally indicates that the original use of HostIP will remain unchanged, +and HostIPs is added for subsequent use + +``` go + // HostIP address information for entries in the (plural) HostIPs field. + // Each entry includes: + // IP: An IP address allocated to the host. + type HostIP struct { + // ip is an IP address (IPv4 or IPv6) assigned to the host + IP string `json:"ip,omitempty" protobuf:"bytes,1,opt,name=ip"` + } + + // HostIPs holds the IP addresses allocated to the host. If this field is specified, the 0th entry must + // match the hostIP field. This list is empty if no IPs have been allocated yet. + // +optional + // +patchStrategy=merge + // +patchMergeKey=ip + HostIPs []HostIP `json:"hostIPs,omitempty" protobuf:"bytes,14,rep,name=hostIPs" patchStrategy:"merge" patchMergeKey:"ip"` +``` + +### Maintaining Compatible Interworking between Old and New Clients + +[See the making a singular field plural](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md#making-a-singular-field-plural) + +### Container Environment Variables + +The Downward API [status.hostIP](https://kubernetes.io/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information/#capabilities-of-the-downward-api) +will preserve the existing single IP address, and will be set to the default IP for each pod. +A new pod API field named `status.hostIPs` will contain a list of IP addresses. +The new pod API will have a slice of structures for the additional IP addresses. +Kubelet will translate the pod structures and return `status.hostIPs` as a comma-delimited string. + +Here is an example of how to define a pluralized `MY_HOST_IPS` environmental +variable in a pod definition yaml file: + +``` yaml +- name: MY_HOST_IPS + valueFrom: + fieldRef: + fieldPath: status.hostIPs +``` + +This definition will cause an environmental variable setting in the pod similar +to the following: + +``` +# PodHostIPs FeatureGate is enabled +MY_HOST_IPS=fd00:10:20:0:3::3,10.20.3.3 + +# PodHostIPs FeatureGate is disabled +MY_HOST_IPS= +``` + +### Test Plan + +Test whether FeatureGate behaves as expected when it is turned on or off +Test whether Downward API supports `status.hostIPs` + +#### Expected behavior + +- If PodHostIPs FeatureGate is enabled: + - `status.hostIPs` there will be all host IPs (IPv4 and IPv6) +- Else: + - `status.hostIPs` will be empty and the ApiServer will reject Downward API `status.hostIPs`, + if Pods already existing Downward API `status.hostIPs`, ensure to ignore it and not affect others. + +### Graduation Criteria + +#### Alpha + +- Feature implemented behind a feature flag + - Add `status.hostIPs` to the PodStatus API + - Downward API support for `status.hostIPs` +- Basic units and e2e tests completed and enabled + +#### Alpha -> Beta Graduation + +- Gather feedback from developers and users +- Expand the e2e tests with more scenarios +- Tests are in Testgrid and linked in the KEP + +#### Beta -> GA Graduation + +- 2 examples of end users using this field +- Allowing time for feedback + +#### Removing a Deprecated Flag + +- Announce deprecation and support policy of the existing flag +- Two versions passed since introducing the functionality that deprecates the flag (to address version skew) +- Address feedback on usage/changed behavior, provided on GitHub issues +- Deprecate the flag + +### Upgrade / Downgrade Strategy + +N/A + +### Version Skew Strategy + +N/A + +## Production Readiness Review Questionnaire + +### Feature Enablement and Rollback + +###### How can this feature be enabled / disabled in a live cluster? + +- [x] Feature gate (also fill in values in `kep.yaml`) + - Feature gate name: PodHostIPs + - Components depending on the feature gate: `kube-apiserver`, `kubelet` +- [ ] Other + - Describe the mechanism: + - Will enabling / disabling the feature require downtime of the control + plane? + - Will enabling / disabling the feature require downtime or reprovisioning + of a node? (Do not assume `Dynamic Kubelet Config` feature is enabled). + +###### Does enabling the feature change any default behavior? + +No. + +###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)? + +Yes. Using the featuregate is the only way to enable/disable this feature. + +###### What happens if we reenable the feature if it was previously rolled back? + +The feature should continue to work just fine. + +###### Are there any tests for feature enablement/disablement? + +No, these will be introduced in the Alpha phase. + +### Rollout, Upgrade and Rollback Planning + +###### How can a rollout or rollback fail? Can it impact already running workloads? + +If the dependent [KEP 563-dual-stack](https://github.com/kubernetes/enhancements/issues/563) is wrong, or could not get IP of host, or setting the field is crashing, this feature may fail to rollout/rollback. +The field is only informative, it doesn't affect running workloads. + +###### What specific metrics should inform a rollback? + +The `status.hostIPs` field in Pod is empty, or frequently updated, or cause any other to crash. + +###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested? + +TBD. + +###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.? + +No. + +### Monitoring Requirements + +###### How can an operator determine if the feature is in use by workloads? + +Pod has a `status.hostIPs` field and use it in downwardAPI to expose it. + +###### How can someone using this feature know that it is working for their instance? + +- [ ] Events + - Event Reason: +- [x] API .status + - Other field: `pod.status.hostIPs` is not empty. +- [ ] Other (treat as last resort) + - Details: + +###### What are the reasonable SLOs (Service Level Objectives) for the enhancement? + +- The feature is only informative, no increased failure rates during use the feature. + +###### What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service? + +- TBD + +###### Are there any missing metrics that would be useful to have to improve observability of this feature? + +No additional metrics needed for this new API field. + +### Dependencies + +###### Does this feature depend on any specific services running in the cluster? + +N/A + +### Scalability + +###### Will enabling / using this feature result in any new API calls? + +No + +###### Will enabling / using this feature result in introducing new API types? + +No + +###### Will enabling / using this feature result in any new calls to the cloud provider? + +No + +###### Will enabling / using this feature result in increasing size or count of the existing API objects? + +Yes. + +- API type(s): Pod +- Estimated increase in size: + - New field in Status about 8 bytes, additional bytes based on whether IPv4(about 4 bytes) or IPv6(about 16 bytes) exists + +###### Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs? + +No + +###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components? + +No + +### Troubleshooting + +N/A + +## Implementation History + +- 2021-05-06: Initial KEP + +## Drawbacks + +N/A + +## Alternatives + +N/A + +## Infrastructure Needed (Optional) + +N/A diff --git a/keps/sig-network/2681-pod-host-ip/kep.yaml b/keps/sig-network/2681-pod-host-ip/kep.yaml new file mode 100644 index 00000000000..e08ac84cafc --- /dev/null +++ b/keps/sig-network/2681-pod-host-ip/kep.yaml @@ -0,0 +1,47 @@ +title: Field status.hostIPs added for Pod +kep-number: 2681 +authors: + - "@wzshiming" +owning-sig: sig-network +participating-sigs: + - sig-node +status: implementable +creation-date: 2021-05-06 +reviewers: + - "@thockin" + - "@khenidak" + - "@danwinship" + - "@aojea" +approvers: + - "@thockin" + - "@khenidak" +see-also: + - "/keps/sig-network/563-dual-stack" +replaces: null + +# The target maturity stage in the current dev cycle for this KEP. +stage: alpha + +# The most recent milestone for which work toward delivery of this KEP has been +# done. This can be the current (upcoming) milestone, if it is being actively +# worked on. +latest-milestone: "v1.24" + +# The milestone at which this feature was, or is targeted to be, at each stage. +milestone: + alpha: "v1.24" + beta: "v1.25" + stable: "v1.27" + +# The following PRR answers are required at alpha release +# List the feature gate name and the components for which it must be enabled +feature-gates: + - name: PodHostIPs + components: + - kube-apiserver + - kubelet +disable-supported: true + +# The following PRR answers are required at beta release +metrics: + - N/A