diff --git a/keps/sig-storage/3751-volume-attributes-class/README.md b/keps/sig-storage/3751-volume-attributes-class/README.md index c79159cef139..c5f4888567cd 100644 --- a/keps/sig-storage/3751-volume-attributes-class/README.md +++ b/keps/sig-storage/3751-volume-attributes-class/README.md @@ -177,22 +177,27 @@ type PersistentVolumeClaimStatus struct { ... // The VolumeAttributesClassName string cannot be empty VolumeAttributesClassName string - ModifyVolumeStatus *PersistentVolumeClaimModifyVolumeStatus + ModifyVolumeStatus ModifyVolumeStatus ... } +type ModifyVolumeStatus struct { + ActualClassName string + TargetClassName string + Status *PersistentVolumeClaimModifyVolumeStatus +} + // +enum type PersistentVolumeClaimModifyVolumeStatus string const ( // When modify volume is complete, the empty string is set by modify volume controller. - PersistentVolumeClaimNoModifyVolumeInProgress PersistentVolumeClaimModifyVolumeStatus = "" + PersistentVolumeClaimNoModifyVolumeInSucceeded PersistentVolumeClaimModifyVolumeStatus = "" // State set when modify volume controller starts modifying the volume in control-plane + // Non terminal errors should also keep this status without modification PersistentVolumeClaimControllerModifyVolumeInProgress PersistentVolumeClaimModifyVolumeStatus = "ControllerModifyVolumeInProgress" // State set when modify volume has failed in modify volume controller with a terminal error. - // Transient errors such as timeout should not set this status and should leave ModifyVolumeStatus - // unmodified, so as modify volume controller can resume the volume modification. - PersistentVolumeClaimControllerModifyVolumeFailed PersistentVolumeClaimModifyVolumeStatus = "ControllerModifyVolumeFailed" + PersistentVolumeClaimControllerModifyVolumeInfeasible PersistentVolumeClaimModifyVolumeStatus = "ControllerModifyVolumeInfeasible" ) ``` @@ -208,30 +213,44 @@ The CSI create request will be extended to add mutable parameters. A new Control // ControllerServer is the server API for Controller service. type ControllerServer interface { ... - rpc ControllerModifyVolume (ModifyVolumeRequest) - returns (ModifyVolumeResponse) { + rpc ControllerModifyVolume (ControllerModifyVolumeRequest) + returns (ControllerModifyVolumeResponse) { option (alpha_method) = true; } ... } -message ModifyVolumeRequest { +message ControllerModifyVolumeRequest { // Contains identity information for the existing volume. // This field is REQUIRED. - string volume_id = 1 - // This field is OPTIONAL.This allows the CO to specify the - // mutable parameters to apply. - map mutable_parameters = 2; + string volume_id = 1; + + // Secrets required by plugin to complete modify volume request. + // This field is OPTIONAL. Refer to the `Secrets Requirements` + // section on how to use this field. + map secrets = 2 [(csi_secret) = true]; + + // Plugin specific volume attributes to mutate, passed in as + // opaque key-value pairs. + // This field is REQUIRED. The Plugin is responsible for + // parsing and validating these parameters. COs will treat these + // as opaque. The CO SHOULD specify the intended values of all mutable + // parameters it intends to modify. SPs MUST NOT modify volumes based + // on the absence of keys, only keys that are specified should result + // in modifications to the volume. + map mutable_parameters = 3; } -message ModifyVolumeResponse {} +message ControllerModifyVolumeResponse {} message CreateVolumeRequest { ... // See CreateVolumeRequest.parameters. // This field is OPTIONAL. map parameters = 4; - // This field is OPTIONAL. This allows the CO to specify the - // volume attributes class parameters to apply. + // Plugins MUST treat these + // as if they take precedence over the parameters field. + // This field SHALL NOT be specified unless the SP has the + // MODIFY_VOLUME plugin capability. map mutable_parameters = 8; } ``` @@ -275,6 +294,7 @@ apiVersion: storage.k8s.io/v1alpha1 kind: VolumeAttributesClass metadata: name: silver +driverName: pd.csi.storage.gke.io parameters: iops: "500" throughput: "50MiB/s" @@ -318,6 +338,7 @@ apiVersion: storage.k8s.io/v1alpha1 kind: VolumeAttributesClass metadata: name: silver +driverName: pd.csi.storage.gke.io parameters: iops: "500" throughput: "50MiB/s" @@ -345,6 +366,7 @@ apiVersion: storage.k8s.io/v1alpha1 kind: VolumeAttributesClass metadata: name: gold +driverName: pd.csi.storage.gke.io parameters: iops: "1000" throughput: "100MiB/s" @@ -409,12 +431,23 @@ The resource quota controller is the only component capable of monitoring and re ### 3. Add new statuses in PVC API to indicate changes of VolumeAttributesClass and the status of the ModifyVolume operation. ``` -type VolumeAttributesClassStatus string +type ModifyVolumeStatus struct { + ActualClassName string + TargetClassName string + Status *PersistentVolumeClaimModifyVolumeStatus +} + +// +enum +type PersistentVolumeClaimModifyVolumeStatus string const ( - PersistentVolumeClaimControllerModifyVolumeProgress VolumeAttributesClassStatus = "ControllerModifyVolumeInProgress" - PersistentVolumeClaimControllerModifyVolumePending VolumeAttributesClassStatus = "ControllerModifyVolumePending" - PersistentVolumeClaimControllerModifyVolumeFailed VolumeAttributesClassStatus = "ControllerModifyVolumeFailed" + // When modify volume is complete, the empty string is set by modify volume controller. + PersistentVolumeClaimNoModifyVolumeInSucceeded PersistentVolumeClaimModifyVolumeStatus = "" + // State set when modify volume controller starts modifying the volume in control-plane + // Non terminal errors should also keep this status without modification + PersistentVolumeClaimControllerModifyVolumeInProgress PersistentVolumeClaimModifyVolumeStatus = "ControllerModifyVolumeInProgress" + // State set when modify volume has failed in modify volume controller with a terminal error. + PersistentVolumeClaimControllerModifyVolumeInfeasible PersistentVolumeClaimModifyVolumeStatus = "ControllerModifyVolumeInfeasible" ) ``` ### 4. Add new CSI API ControllerModifyVolume, when there is a change of VolumeAttributesClass in PVC, external-resizer triggers a ControllerModifyVolume operation against a CSI endpoint. A Controller Plugin MUST implement this RPC call if it has MODIFY_VOLUME capability. @@ -454,7 +487,7 @@ There are a few conditions that will trigger add/remove pvc finalizers in the Vo 1. PVC created with a VolumeAttributesClass The **VACObjectInUseProtection admission controller**: - * Check if the VolumeAttributesClass exists. If not, the PVC will enter the PENDING state because we do not want to impose ordering on object creation + * Check if the VolumeAttributesClass exists. If not, the PVC will enter the INPROGRESS state because we do not want to impose ordering on object creation * Check if this VolumeAttributesClass already has a protection finalizer * Add the finalizer to the VolumeAttributesClass if there is none 2. PVC created with a VolumeAttributesClass being deleted @@ -480,7 +513,7 @@ For unbound PVs referencing a VAC: 1. Unbound PV created with a VolumeAttributesClass The **VACObjectInUseProtection admission controller**: - * Check if the VolumeAttributesClass exists. If not, the PV will enter the PENDING state because we do not want to impose ordering on object creation + * Check if the VolumeAttributesClass exists. If not, the PV will enter the INPROGRESS state because we do not want to impose ordering on object creation * Check if this VolumeAttributesClass already has a protection finalizer * Add the finalizer to the VolumeAttributesClass if there is none 2. PV has a VolumeAttributesClass and this PV is deleted @@ -496,6 +529,7 @@ apiVersion: storage.k8s.io/v1alpha1 kind: VolumeAttributesClass metadata: name: silver +driverName: pd.csi.storage.gke.io parameters: iops: "500" throughput: "50MiB/s" @@ -569,6 +603,7 @@ apiVersion: storage.k8s.io/v1alpha1 kind: VolumeAttributesClass metadata: name: gold +driverName: pd.csi.storage.gke.io parameters: iops: "1000" throughput: "100MiB/s" @@ -593,7 +628,7 @@ spec: ModifyVolume is only allowed on bound PVCs. Under the ModifyVolume call, it will pass in the mutable parameters and do the update operation based on the `VolumeAttributesClass` parameters. -![ModifyVolume Flow Diagram](./VolumeAttributesClass-ModifyVolume-Flow.png) +![ModifyVolume Flow Diagram](./VolumeAttributesClass-ModifyVolume-Flow-v2.png) ### Implementation & Handling Failure diff --git a/keps/sig-storage/3751-volume-attributes-class/VolumeAttributesClass-ModifyVolume-Flow-v2.png b/keps/sig-storage/3751-volume-attributes-class/VolumeAttributesClass-ModifyVolume-Flow-v2.png new file mode 100644 index 000000000000..0dfb776a0cad Binary files /dev/null and b/keps/sig-storage/3751-volume-attributes-class/VolumeAttributesClass-ModifyVolume-Flow-v2.png differ diff --git a/keps/sig-storage/3751-volume-attributes-class/VolumeAttributesClass-ModifyVolume-Flow.png b/keps/sig-storage/3751-volume-attributes-class/VolumeAttributesClass-ModifyVolume-Flow.png deleted file mode 100644 index 5b39e7e9a229..000000000000 Binary files a/keps/sig-storage/3751-volume-attributes-class/VolumeAttributesClass-ModifyVolume-Flow.png and /dev/null differ