From aa677488bd8f2e6fc4e2ba12411d191909b99558 Mon Sep 17 00:00:00 2001 From: rajkumar-palani <43825650+rajkumar-palani@users.noreply.github.com> Date: Wed, 22 Feb 2023 12:59:31 +0530 Subject: [PATCH] CSM Docs v1.5.1 Changes - Release 1.5.1 to Main (#485) --- config.toml | 2 +- content/docs/_index.md | 10 +- content/docs/authorization/_index.md | 2 + .../configuration/powerflex/_index.md | 4 +- .../authorization/deployment/rpm/_index.md | 2 +- content/docs/authorization/release/_index.md | 19 +--- content/docs/csidriver/_index.md | 4 +- .../csidriver/installation/helm/powerstore.md | 2 +- content/docs/csidriver/release/powerstore.md | 10 +- .../upgradation/drivers/powerstore.md | 21 +---- content/docs/references/cli/_index.md | 91 ++++++++++++++++++- 11 files changed, 119 insertions(+), 48 deletions(-) diff --git a/config.toml b/config.toml index 3816fce2..55dee884 100644 --- a/config.toml +++ b/config.toml @@ -169,7 +169,7 @@ enable = false # icon = "fa fa-envelope" # desc = "Discuss development issues around the project" [[params.versions]] - version = "Current(v1.5)" + version = "Current(v1.5.1)" url = "https://dell.github.io/csm-docs/docs/" [[params.versions]] diff --git a/content/docs/_index.md b/content/docs/_index.md index e986d91c..e982c146 100644 --- a/content/docs/_index.md +++ b/content/docs/_index.md @@ -64,9 +64,9 @@ CSM is made up of multiple components including modules (enterprise capabilities ## CSM Supported Modules and Dell CSI Drivers -| Modules/Drivers | CSM 1.5 | [CSM 1.4](../v1/) | [CSM 1.3.1](../v2/) | [CSM 1.2.1](../v3/) | +| Modules/Drivers | CSM 1.5.1 | [CSM 1.4](../v1/) | [CSM 1.3.1](../v2/) | [CSM 1.2.1](../v3/) | | -------------------------------------------------------------------------------------------- | :-----: | :---------------: | :---------------: | :-----------------: | -| [Authorization](https://hub.docker.com/r/dellemc/csm-authorization-sidecar) | v1.5.0 | v1.4.0 | v1.3.0 | v1.2.0 | +| [Authorization](https://hub.docker.com/r/dellemc/csm-authorization-sidecar) | v1.5.1 | v1.4.0 | v1.3.0 | v1.2.0 | | [Observability](https://hub.docker.com/r/dellemc/csm-topology) | v1.4.0 | v1.3.0 | v1.2.0 | v1.1.1 | | [Replication](https://hub.docker.com/r/dellemc/dell-replication-controller) | v1.3.1 | v1.3.0 | v1.3.0 | v1.2.0 | | [Resiliency](https://hub.docker.com/r/dellemc/podmon) | v1.4.0 | v1.3.0 | v1.2.0 | v1.1.0 | @@ -74,15 +74,15 @@ CSM is made up of multiple components including modules (enterprise capabilities | [Application Mobility](https://hub.docker.com/r/dellemc/csm-application-mobility-controller) | v0.2.0 | v0.1.0 | NA | NA | | [CSI Driver for PowerScale](https://hub.docker.com/r/dellemc/csi-isilon/tags) | v2.5.0 | v2.4.0 | v2.3.0 | v2.2.0 | | [CSI Driver for Unity XT](https://hub.docker.com/r/dellemc/csi-unity/tags) | v2.5.0 | v2.4.0 | v2.3.0 | v2.2.0 | -| [CSI Driver for PowerStore](https://hub.docker.com/r/dellemc/csi-powerstore/tags) | v2.5.0 | v2.4.0 | v2.3.0 | v2.2.0 | +| [CSI Driver for PowerStore](https://hub.docker.com/r/dellemc/csi-powerstore/tags) | v2.5.1 | v2.4.0 | v2.3.0 | v2.2.0 | | [CSI Driver for PowerFlex](https://hub.docker.com/r/dellemc/csi-vxflexos/tags) | v2.5.0 | v2.4.0 | v2.3.0 | v2.2.0 | | [CSI Driver for PowerMax](https://hub.docker.com/r/dellemc/csi-powermax/tags) | v2.5.0 | v2.4.0 | v2.3.1 | v2.2.0 | ## CSM Modules Support Matrix for Dell CSI Drivers -| CSM Module | CSI PowerFlex v2.5.0 | CSI PowerScale v2.5.0 | CSI PowerStore v2.5.0 | CSI PowerMax v2.5.0 | CSI Unity XT v2.5.0 | +| CSM Module | CSI PowerFlex v2.5.0 | CSI PowerScale v2.5.0 | CSI PowerStore v2.5.1 | CSI PowerMax v2.5.0 | CSI Unity XT v2.5.0 | | ----------------------------------------------------------- | -------------------- | --------------------- | --------------------- | ------------------- | ------------------- | -| [**Authorization**](authorization/) v1.5 | ✔️ | ✔️ | ❌ | ✔️ | ❌ | +| [**Authorization**](authorization/) v1.5.1 | ✔️ | ✔️ | ❌ | ✔️ | ❌ | | [**Observability**](observability/) v1.4 | ✔️ | ✔️ | ✔️ | ❌ | ❌ | | [**Replication**](replication/) v1.3.1 | ❌ | ✔️ | ✔️ | ✔️ | ❌ | | [**Resiliency**](resiliency/) v1.4 | ✔️ | ✔️ | ❌ | ❌ | ✔️ | diff --git a/content/docs/authorization/_index.md b/content/docs/authorization/_index.md index 33a7425e..12808a5a 100644 --- a/content/docs/authorization/_index.md +++ b/content/docs/authorization/_index.md @@ -70,6 +70,8 @@ CSM for Authorization consists of 2 components - the Authorization sidecar and t | dellemc/csm-authorization-sidecar:v1.2.0 | v1.1.0, v1.2.0 | | dellemc/csm-authorization-sidecar:v1.3.0 | v1.1.0, v1.2.0, v1.3.0 | | dellemc/csm-authorization-sidecar:v1.4.0 | v1.1.0, v1.2.0, v1.3.0, v1.4.0 | +| dellemc/csm-authorization-sidecar:v1.5.0 | v1.1.0, v1.2.0, v1.3.0, v1.4.0, v1.5.0 | +| dellemc/csm-authorization-sidecar:v1.5.1 | v1.1.0, v1.2.0, v1.3.0, v1.4.0, v1.5.0, v1.5.1 | {{}} ## Roles and Responsibilities diff --git a/content/docs/authorization/configuration/powerflex/_index.md b/content/docs/authorization/configuration/powerflex/_index.md index 290e3cc9..d3f122dd 100644 --- a/content/docs/authorization/configuration/powerflex/_index.md +++ b/content/docs/authorization/configuration/powerflex/_index.md @@ -58,4 +58,6 @@ Create the karavi-authorization-config secret using this command: Enable CSM for Authorization and provide the *proxyHost* address -7. Install the CSI PowerFlex driver \ No newline at end of file +7. Install the CSI PowerFlex driver + +8. (Optional) Install [dellctl](../../../references/cli) to perform Kubernetes administrator commands for additional capabilities (e.g., list volumes). Please refer to the [dellctl documentation page](../../../references/cli) for the installation steps and command list. \ No newline at end of file diff --git a/content/docs/authorization/deployment/rpm/_index.md b/content/docs/authorization/deployment/rpm/_index.md index 07897601..ddddc0ed 100644 --- a/content/docs/authorization/deployment/rpm/_index.md +++ b/content/docs/authorization/deployment/rpm/_index.md @@ -46,7 +46,7 @@ If CSM for Authorization is being installed on a system where SELinux is enabled ### Shell Script Installer -As of version 1.5.0, the easiest way to obtain the tar archive with the shell script installer is directly from the [GitHub repository's releases](https://github.com/dell/karavi-authorization/releases) section. +The easiest way to obtain the tar archive with the shell script installer is directly from the [GitHub repository's releases](https://github.com/dell/karavi-authorization/releases) section. Alternatively, the tar archive can be built from source by cloning the [GitHub repository](https://github.com/dell/karavi-authorization) and using the following Makefile targets to build the installer: diff --git a/content/docs/authorization/release/_index.md b/content/docs/authorization/release/_index.md index 4352059f..16ff0da3 100644 --- a/content/docs/authorization/release/_index.md +++ b/content/docs/authorization/release/_index.md @@ -6,22 +6,13 @@ Description: > Dell Container Storage Modules (CSM) release notes for authorization --- -## Release Notes - CSM Authorization 1.4.0 +## Release Notes - CSM Authorization 1.5.1 ### New Features/Changes -- CSM 1.4 Release specific changes. ([#350](https://github.com/dell/csm/issues/350)) -- CSM Authorization insecure related entities are renamed to skipCertificateValidation. ([#368](https://github.com/dell/csm/issues/368)) +- Show volumes associated with the tenant from the k8s server. ([#408](https://github.com/dell/csm/issues/408)) +- CSM 1.5.1 release specific changes. ([#582](https://github.com/dell/csm/issues/582)) -### Bugs +### Bugs -- PowerScale volumes unable to be created with Helm deployment of CSM Authorization. ([#419](https://github.com/dell/csm/issues/419)) -- Authorization CLI documentation does not mention --array-insecure flag when creating or updating storage systems. ([#416](https://github.com/dell/csm/issues/416)) -- Authorization: Add documentation for backing up and restoring redis data. ([#410](https://github.com/dell/csm/issues/410)) -- CSM Authorization doesn't recognize storage with capital letters. ([#398](https://github.com/dell/csm/issues/398)) -- Update Authorization documentation with supported versions of k3s-selinux and container-selinux packages. ([#393](https://github.com/dell/csm/issues/393)) -- Using Authorization without dependency on jq. ([#390](https://github.com/dell/csm/issues/390)) -- Authorization Documentation Improvement. ([#384](https://github.com/dell/csm/issues/384)) -- Unit test failing for csm-authorization. ([#382](https://github.com/dell/csm/issues/382)) -- Karavictl has incorrect permissions after download. ([#360](https://github.com/dell/csm/issues/360)) -- Helm deployment of Authorization denies a valid request path from csi-powerflex. ([#353](https://github.com/dell/csm/issues/353)) \ No newline at end of file +- CSM Authorization installation fails due to a PATH not looking in /usr/local/bin. ([#580](https://github.com/dell/csm/issues/580)) diff --git a/content/docs/csidriver/_index.md b/content/docs/csidriver/_index.md index c4851b08..8f3b093e 100644 --- a/content/docs/csidriver/_index.md +++ b/content/docs/csidriver/_index.md @@ -33,14 +33,14 @@ The CSI Drivers by Dell implement an interface between [CSI](https://kubernetes- {{}} | Features | PowerMax | PowerFlex | Unity XT | PowerScale | PowerStore | |--------------------------|:--------:|:---------:|:---------:|:----------:|:----------:| -| CSI Driver version | 2.5.0 | 2.5.0 | 2.5.0 | 2.5.0 | 2.5.0 | +| CSI Driver version | 2.5.0 | 2.5.0 | 2.5.0 | 2.5.0 | 2.5.1 | | Static Provisioning | yes | yes | yes | yes | yes | | Dynamic Provisioning | yes | yes | yes | yes | yes | | Expand Persistent Volume | yes | yes | yes | yes | yes | | Create VolumeSnapshot | yes | yes | yes | yes | yes | | Create Volume from Snapshot | yes | yes | yes | yes | yes | | Delete Snapshot | yes | yes | yes | yes | yes | -| [Access Mode](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes)| **FC/iSCSI:**
RWO/
RWOP
**Raw block:**
RWO/
RWX/
ROX/
RWOP | RWO/ROX/RWOP

RWX (Raw block only) | RWO/ROX/RWOP

RWX (Raw block & NFS only) | RWO/RWX/ROX/
RWOP | RWO/RWOP
(FC/iSCSI)
RWO/
RWX/
ROX/
RWOP
(RawBlock, NFS) | +| [Access Mode](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#access-modes)| **FC/iSCSI/NVMe:**
RWO/
RWOP
**Raw block:**
RWO/
RWX/
ROX/
RWOP | RWO/ROX/RWOP

RWX (Raw block only) | RWO/ROX/RWOP

RWX (Raw block & NFS only) | RWO/RWX/ROX/
RWOP | RWO/RWOP
(FC/iSCSI)
RWO/
RWX/
ROX/
RWOP
(RawBlock, NFS) | | CSI Volume Cloning | yes | yes | yes | yes | yes | | CSI Raw Block Volume | yes | yes | yes | no | yes | | CSI Ephemeral Volume | no | yes | yes | yes | yes | diff --git a/content/docs/csidriver/installation/helm/powerstore.md b/content/docs/csidriver/installation/helm/powerstore.md index e1c84927..17269373 100644 --- a/content/docs/csidriver/installation/helm/powerstore.md +++ b/content/docs/csidriver/installation/helm/powerstore.md @@ -177,7 +177,7 @@ CRDs should be configured during replication prepare stage with repctl as descri ## Install the Driver **Steps** -1. Run `git clone -b v2.5.0 https://github.com/dell/csi-powerstore.git` to clone the git repository. +1. Run `git clone -b v2.5.1 https://github.com/dell/csi-powerstore.git` to clone the git repository. 2. Ensure that you have created namespace where you want to install the driver. You can run `kubectl create namespace csi-powerstore` to create a new one. "csi-powerstore" is just an example. You can choose any name for the namespace. But make sure to align to the same namespace during the whole installation. 3. Edit `samples/secret/secret.yaml` file and configure connection information for your PowerStore arrays changing following parameters: diff --git a/content/docs/csidriver/release/powerstore.md b/content/docs/csidriver/release/powerstore.md index 5594f09e..ebf09d1f 100644 --- a/content/docs/csidriver/release/powerstore.md +++ b/content/docs/csidriver/release/powerstore.md @@ -3,18 +3,15 @@ title: PowerStore description: Release notes for PowerStore CSI driver --- -## Release Notes - CSI PowerStore v2.5.0 +## Release Notes - CSI PowerStore v2.5.1 ### New Features/Changes -- [Added support for Storage Capacity Tracking feature](https://github.com/dell/csm/issues/483) -- [Added support for Kubernetes 1.25](https://github.com/dell/csm/issues/478) -- [Added support for OpenShift 4.11](https://github.com/dell/csm/issues/480) -- [Added support for PowerStore 3.2 array](https://github.com/dell/csm/issues/482) +There are no features/changes in this release. ### Fixed Issues -There are no fixed issues in this release. +- [Fixed issue where driver was not properly cleaning up resources when volumes were unmounted](https://github.com/dell/csm/issues/666) ### Known Issues @@ -28,3 +25,4 @@ There are no fixed issues in this release. ### Note: - Support for Kubernetes alpha features like Volume Health Monitoring and RWOP (ReadWriteOncePod) access mode will not be available in Openshift environment as Openshift doesn't support enabling of alpha features for Production Grade clusters. +- This release is only supported when driver is installed via helm. \ No newline at end of file diff --git a/content/docs/csidriver/upgradation/drivers/powerstore.md b/content/docs/csidriver/upgradation/drivers/powerstore.md index 4bac7a47..31175162 100644 --- a/content/docs/csidriver/upgradation/drivers/powerstore.md +++ b/content/docs/csidriver/upgradation/drivers/powerstore.md @@ -7,15 +7,15 @@ weight: 1 Description: Upgrade PowerStore CSI driver --- -You can upgrade the CSI Driver for Dell PowerStore using Helm or Dell CSI Operator. +You can upgrade the CSI Driver for Dell PowerStore using Helm. -## Update Driver from v2.4 to v2.5 using Helm +## Update Driver from v2.5 to v2.5.1 using Helm Note: While upgrading the driver via helm, controllerCount variable in myvalues.yaml can be at most one less than the number of worker nodes. **Steps** -1. Run `git clone -b v2.5.0 https://github.com/dell/csi-powerstore.git` to clone the git repository and get the driver. -2. Edit `helm/config.yaml` file and configure connection information for your PowerStore arrays changing the following parameters: +1. Run `git clone -b v2.5.1 https://github.com/dell/csi-powerstore.git` to clone the git repository and get the driver. +2. Edit `samples/secret/secret.yaml` file and configure connection information for your PowerStore arrays changing the following parameters: - *endpoint*: defines the full URL path to the PowerStore API. - *globalID*: specifies what storage cluster the driver should use - *username*, *password*: defines credentials for connecting to array. @@ -28,18 +28,7 @@ Note: While upgrading the driver via helm, controllerCount variable in myvalues. Add more blocks similar to above for each PowerStore array if necessary. 3. (optional) create new storage classes using ones from `samples/storageclass` folder as an example and apply them to the Kubernetes cluster by running `kubectl create -f ` - >Storage classes created by v1.4/v2.0/v2.1/v2.2/v2.3/v2.4 driver will not be deleted, v2.5 driver will use default array to manage volumes provisioned with old storage classes. Thus, if you still have volumes provisioned by v1.4/v2.0/v2.1/v2.2/v2.3/v2.4 in your cluster then be sure to include the same array you have used for the v1.4/v2.0/v2.1/v2.2/v2.3/v2.4 driver and make it default in the `config.yaml` file. + >Storage classes created by v1.4/v2.0/v2.1/v2.2/v2.3/v2.4/v2.5 driver will not be deleted, v2.5.1 driver will use default array to manage volumes provisioned with old storage classes. Thus, if you still have volumes provisioned by v1.4/v2.0/v2.1/v2.2/v2.3/v2.4/v2.5 in your cluster then be sure to include the same array you have used for the v1.4/v2.0/v2.1/v2.2/v2.3/v2.4/v2.5 driver and make it default in the `secret.yaml` file. 4. Create the secret by running ```kubectl create secret generic powerstore-config -n csi-powerstore --from-file=config=secret.yaml``` 5. Copy the default values.yaml file `cd dell-csi-helm-installer && cp ../helm/csi-powerstore/values.yaml ./my-powerstore-settings.yaml` and update parameters as per the requirement. 6. Run the `csi-install` script with the option _\-\-upgrade_ by running: `./csi-install.sh --namespace csi-powerstore --values ./my-powerstore-settings.yaml --upgrade`. - -## Upgrade using Dell CSI Operator: - -**Notes:** -1. While upgrading the driver via operator, replicas count in sample CR yaml can be at most one less than the number of worker nodes. -2. Upgrading the Operator does not upgrade the CSI Driver. - - -1. Please upgrade the Dell CSI Operator by following [here](./../operator). -2. Once the operator is upgraded, to upgrade the driver, refer [here](./../../../installation/operator/#update-csi-drivers). - diff --git a/content/docs/references/cli/_index.md b/content/docs/references/cli/_index.md index 79121293..d631a60a 100644 --- a/content/docs/references/cli/_index.md +++ b/content/docs/references/cli/_index.md @@ -30,10 +30,12 @@ This document outlines all dellctl commands, their intended use, options that ca | [dellctl schedule get](#dellctl-schedule-get) | Get schedules | | [dellctl encryption rekey](#dellctl-encryption-rekey) | Rekey an encrypted volume | | [dellctl encryption rekey-status](#dellctl-encryption-rekey-status) | Get status of an encryption rekey operation | +| [dellctl images](#dellctl-images) | List the container images needed by csi driver | +| [dellctl volume get](#dellctl-volume-get) | Gets PowerFlex volume infomation for a given tenant on a local cluster | ## Installation instructions -1. Download `dellctl` from [here](https://github.com/dell/csm/releases/tag/v1.4.0). +1. Download `dellctl` from [here](https://github.com/dell/csm/releases/tag/v1.5.1). 2. chmod +x dellctl 3. Move `dellctl` to `/usr/local/bin` or add `dellctl`'s containing directory path to PATH environment variable. 4. Run `dellctl --help` to know available commands or run `dellctl command --help` to know more about a specific command. @@ -782,3 +784,90 @@ Encryption rekey status with name of the rekey object # dellctl encryption rekey-status myrekey INFO Status of rekey request myrekey = completed ``` + +### dellctl images + +List the container images needed by csi driver + +**NOTE.**: dellctl images currently supports csi-vxflexos driver only. + +#### Aliases + +``` +images,imgs +``` + +#### Flags + +``` + Flags: + -d, --driver string csi driver name + -h, --help help for images + +``` +#### Output + + +``` +# dellctl images --driver csi-vxflexos +Driver Image Supported Orchestrator Versions Sidecar Images +dellemc/csi-vxflexos:v2.5.0 k8s1.25,k8s1.24,k8s1.23,ocp4.11,ocp4.10 k8s.gcr.io/sig-storage/csi-attacher:v4.0.0 + k8s.gcr.io/sig-storage/csi-provisioner:v3.3.0 + dellemc/csi-volumegroup-snapshotter:v1.2.0 + k8s.gcr.io/sig-storage/csi-external-health-monitor-controller:v0.7.0 + k8s.gcr.io/sig-storage/csi-snapshotter:v6.1.0 + k8s.gcr.io/sig-storage/csi-resizer:v1.6.0 + k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.6.0 + dellemc/sdc:3.6.0.6 + +dellemc/csi-vxflexos:v2.4.0 k8s1.24,k8s1.23,k8s1.22,ocp4.10,ocp4.9 k8s.gcr.io/sig-storage/csi-attacher:v3.5.0 + k8s.gcr.io/sig-storage/csi-provisioner:v3.2.1 + dellemc/csi-volumegroup-snapshotter:v1.2.0 + k8s.gcr.io/sig-storage/csi-external-health-monitor-controller:v0.6.0 + k8s.gcr.io/sig-storage/csi-snapshotter:v6.0.1 + k8s.gcr.io/sig-storage/csi-resizer:v1.5.0 + k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.5.1 + dellemc/sdc:3.6.0.6 + +dellemc/csi-vxflexos:v2.3.0 k8s1.24,k8s1.23,k8s1.22,ocp4.10,ocp4.9 k8s.gcr.io/sig-storage/csi-attacher:v3.4.0 + k8s.gcr.io/sig-storage/csi-provisioner:v3.1.0 + dellemc/csi-volumegroup-snapshotter:v1.0.1 + gcr.io/k8s-staging-sig-storage/csi-external-health-monitor-controller:v0.5.0 + k8s.gcr.io/sig-storage/csi-snapshotter:v5.0.1 + k8s.gcr.io/sig-storage/csi-resizer:v1.4.0 + k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.5.1 + dellemc/sdc:3.6.0.6 +``` + + +### dellctl volume get + +Gets PowerFlex volume infomation for a given tenant on a local cluster + +##### Aliases + get, ls, list + +##### Flags + +``` + -h, --help help for get + --insecure optionalBool[=true] provide flag to skip certificate validation + --namespace string namespace of the secret for the given tenant + --proxy string auth proxy endpoint to use +``` + +##### Output + +Gets PowerFlex volume infomation for a given tenant on a local cluster. The namespace is the namespace where tenant secret is created. + +>Note: This was output was generated using Authorization Proxy version 1.5.1. Please ensure you are using version 1.5.1 or greater. + +``` +# dellctl volume get --proxy --namespace vxflexos +NAME VOLUME ID SIZE POOL SYSTEM ID PV NAME PV STATUS STORAGE CLASS PVC NAME NAMESPACE +k8s-e7c8b39112 a69bf18e00000008 8.000000 mypool 636468e3638c840f k8s-e7c8b39112 Released vxflexos demo-claim10 default +k8s-e6e2b46103 a69bf18f00000009 8.000000 mypool 636468e3638c840f k8s-e6e2b46103 Bound vxflexos demo-claim11 default +k8s-b1abb817d3 a69bf19000000001 8.000000 mypool 636468e3638c840f k8s-b1abb817d3 Bound vxflexos demo-claim13 default +k8s-28e4184f41 c6b2280d0000009a 8.000000 mypool 636468e3638c840f k8s-28e4184f41 Available local-storage +k8s-7296621062 a69b554f00000004 8.000000 mypool 636468e3638c840f +```