From 602366509d3629b13d309c59116550620c7242b2 Mon Sep 17 00:00:00 2001 From: Sergey Berezansky <66549633+sergeyberezansky@users.noreply.github.com> Date: Sun, 2 Apr 2023 11:14:07 +0300 Subject: [PATCH] docs: Improve documentation on mount options and different volume types (#39) * docs: create custom mount options README.md * docs: append readonly exam * docs: append comparison of different volume types to usage.md --- docs/usage.md | 42 +++++++++++++++++++ examples/dynamic_filesystem/README.md | 1 + .../csi-app-on-fs-api-readonly.yaml | 21 ++++++++++ examples/mount_options/README.md | 22 ++++++++++ .../csi-app-on-fs-mountoptions.yaml | 20 +++++++++ .../pvc-wekafs-fs-mountoptions.yaml | 12 ++++++ .../storageclass-wekafs-mountoptions.yaml | 2 +- 7 files changed, 119 insertions(+), 1 deletion(-) create mode 100644 examples/dynamic_filesystem/csi-app-on-fs-api-readonly.yaml create mode 100644 examples/mount_options/README.md create mode 100644 examples/mount_options/csi-app-on-fs-mountoptions.yaml create mode 100644 examples/mount_options/pvc-wekafs-fs-mountoptions.yaml diff --git a/docs/usage.md b/docs/usage.md index 8735107fc..a91a78196 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -23,6 +23,48 @@ The logic in dynamic provisioning is as following: > **NOTE:** The example above is based on directory-backed CSI volume, but same logic applies also to other volume > types. +### Choosing the right volume type for your workload +Weka CSI plugin now supports multiple types of volumes, which basically differ by their representation on Weka cluster backend. +Each volume type has its benefits and limitations. It is crucial to choose the right type of volume to achieve the best +manageability and performance. + +| Volume type | Directory-backed volume | Snapshot-backed volume | Filesystem-backed volume | Hybrid volume | +|---------------------------------------------|-----------------------------------------------|----------------------------------------------------------------|------------------------------------------------------------------------------|----------------------------------------------------------------| +| Representation on Weka storage | Directory with attached quota | Writable snapshot with attached quota | Filesystem with attached quota | Directory inside writable snapshot with attached quota | +| `filesystemName` in storageClass | Must be set | Must be set | | Storageclass is inherited from parent volume | +| `filesystemGroupName` in storageClass | - | - | Must be set | - | +| Requires pre-configured filesystem | Yes | Yes | - | Located on same filesystem where source content is | +| Dynamic provision support | Yes | Yes | Yes | Yes | +| Static provision support | Yes | Yes | Yes | Yes | +| Max. number of volumes | Unlimited | Limited by max. number of writable snapshots1 | Limited by max. number of filesystems2 | Limited by max. number of writable snapshots1 | +| CSI Snapshot Support | Supported, Off by default3 | Supported | Supported | Supported, Off by default3 | +| CSI ContentSource: Snapshot | Supported | Supported | Supported | Supported | +| CSI ContentSource: Volume | Supported | Supported | Supported | Supported | +| Minimum supported Weka version | 3.13 and up | 3.14-4.1 with limitations4
4.2 fully supported | 3.14 and up | 3.14-4.1 with limitations4
4.2 fully supported | +| Custom mountOptions | Per storage class | Per storage class | Per storage class | Per storage class | +| Shared caching (on Kubernetes worker) | across all volumes on a filesystem | across all volumes on a filesystem | per volume | across all volumes on a filesystem | +| SELinux support | Yes | Yes | Yes | Yes | +| Volume expansion support | Yes | Yes | Yes | Yes | +| Capacity allocation | Shared5 | Shared5 | Dedicated | Shared5 | +| Can be backed up externally (E.g. snap2Obj) | Yes (a whole filesystem) | No | Yes6 | No | +| Can be tiered to OBS (outside CSI) | Yes (a whole filesystem) | No | Yes6 | No | +| Snapshot can be uploaded to OBS | Yes (a whole filesystem) | No | Yes6 | No | +| Recommended usage pattern | Millions of small, thinly provisioned volumes | Thousands of volumes with mostly appended data | Hundreds of volumes with performance and consistent backup being key factors | Thousands of volumes created from same contentSource | + +Comments: +1. Number of writable snapshots differs between different versions of Weka software. Please refer to documentation for your particular installed version for additional information +2. Number of filesystems differs between different versions of Weka software. Please refer to documentation for your particular installed version for additional information +3. CSI snapshot of directory-backed volume creates a snapshot of the whole filesystem on which the directory is located. + As a result, the capacity required by such snapshot would significantly depend on data usage pattern of all CSI directory-backed volumes on same filesystem, and much larger than the volume size. + Hence, snapshot creation is prohibited by default, but can be enabled. Refer to Weka CSI Plugin Helm chart documentation for additional information +4. In Weka versions prior to 4.2, quota is not enforced inside filesystem snapshots. As a result, capacity enforcement is not supported for this type of volume. + If capacity enforcement is crucial for your workload, use directory-backed volumes or upgrade to latest Weka software +5. Filesystem size and used capacity is not monitoried by CSI plugin. The administrator has to make sure enough capacity is allocated for the filesystem. + Filesystems created dynamically (via filesystem-backed volume) can be set with initial size to accomodate future volumes, refer to Weka CSI Plugin Helm chart documentation for additional information +6. Weka CSI plugin does not support automatic configuration of tiering for filesystem-backed volumes, but those can be set externally. + +#### Directory-based volumes +Directory-based volumes, also called "legacy", are represented by single directory inside For additional information regarding different volume types and how to use them, refer to the following documentation: - Dynamic provisioning of [directory-backed volumes](../examples/dynamic_directory/README.md) diff --git a/examples/dynamic_filesystem/README.md b/examples/dynamic_filesystem/README.md index 93fac7dd2..6fdae45ca 100644 --- a/examples/dynamic_filesystem/README.md +++ b/examples/dynamic_filesystem/README.md @@ -56,3 +56,4 @@ This example introduces automatic provisioning of filesystems. For this function 10. Create application that tails content of `/data/temp.txt` from volume created from snapshot: `csi-app-on-fs-clone` - the file should exist and be accessible - the latest timestamp you are expected to see is the timestamp just before volume cloning +11. Optionally, create another application that access data in read-only mode: `csi-app-on-fs-api-readonly` diff --git a/examples/dynamic_filesystem/csi-app-on-fs-api-readonly.yaml b/examples/dynamic_filesystem/csi-app-on-fs-api-readonly.yaml new file mode 100644 index 000000000..079d938ca --- /dev/null +++ b/examples/dynamic_filesystem/csi-app-on-fs-api-readonly.yaml @@ -0,0 +1,21 @@ +kind: Pod +apiVersion: v1 +metadata: + name: csi-app-on-fs-api +spec: + # make sure that pod is scheduled only on node having weka CSI node running + nodeSelector: + topology.csi.weka.io/global: "true" + containers: + - name: my-frontend + image: ubuntu + volumeMounts: + - mountPath: "/data" + name: my-csi-volume + readOnly: true + command: ["/bin/sh"] + args: ["-c", "while true; do echo `date` hello >> /data/temp.txt; sleep 10;done"] + volumes: + - name: my-csi-volume + persistentVolumeClaim: + claimName: pvc-wekafs-fs-api # defined in pvc-wekafs-dir-api.yaml diff --git a/examples/mount_options/README.md b/examples/mount_options/README.md new file mode 100644 index 000000000..259783a9f --- /dev/null +++ b/examples/mount_options/README.md @@ -0,0 +1,22 @@ +# Overview + +## Example Intentions +In rare circumstances, additional mount options (e.g. different caching settings) are desired for a particular flow. + +1. This example concentrates on setting custom mount options +2. The example makes a use of a filesystem-backed volume, but the feature is functional on any other type of volume backings + +# Workflow +> All commands below may be executed by `kubectl apply -f .yaml` +1. Create storageclass `storageclass-wekafs-mountoptions` +2. Create CSI secret `csi-wekafs-api-secret` (Located in [../common/csi-wekafs-api-secret.yaml](../common/csi-wekafs-api-secret.yaml)) +3. Provision a new volume `pvc-wekafs-fs-mountoptions` +4. Create application that writes timestamp every 10 seconds into `/data/temp.txt`: `csi-app-fs-mountoptions` +5. Attach to the application and validate the options are added correctly + ``` + kubectl exec csi-app-fs-mountoptions -- mount -t wekafs + ``` + The output should resemble this: + ``` + csivol-pvc-15a45f20-Z72GJXDCEWQ5 on /data type wekafs (rw,relatime,readcache,noatime,readahead_kb=32768,dentry_max_age_positive=1000,dentry_max_age_negative=0) + ``` diff --git a/examples/mount_options/csi-app-on-fs-mountoptions.yaml b/examples/mount_options/csi-app-on-fs-mountoptions.yaml new file mode 100644 index 000000000..9d15dc6d5 --- /dev/null +++ b/examples/mount_options/csi-app-on-fs-mountoptions.yaml @@ -0,0 +1,20 @@ +kind: Pod +apiVersion: v1 +metadata: + name: csi-app-on-fs-mountoptions +spec: + # make sure that pod is scheduled only on node having weka CSI node running + nodeSelector: + topology.csi.weka.io/global: "true" + containers: + - name: my-frontend + image: ubuntu + volumeMounts: + - mountPath: "/data" + name: my-csi-volume + command: ["/bin/sh"] + args: ["-c", "while true; do echo `date` hello >> /data/temp.txt; sleep 10;done"] + volumes: + - name: my-csi-volume + persistentVolumeClaim: + claimName: pvc-wekafs-fs-mountoptions # defined in pvc-wekafs-dir-api.yaml diff --git a/examples/mount_options/pvc-wekafs-fs-mountoptions.yaml b/examples/mount_options/pvc-wekafs-fs-mountoptions.yaml new file mode 100644 index 000000000..516186117 --- /dev/null +++ b/examples/mount_options/pvc-wekafs-fs-mountoptions.yaml @@ -0,0 +1,12 @@ +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: pvc-wekafs-fs-mountoptions +spec: + accessModes: + - ReadOnlyMany + storageClassName: storageclass-wekafs-fs-mountoptions + volumeMode: Filesystem + resources: + requests: + storage: 1Gi diff --git a/examples/mount_options/storageclass-wekafs-mountoptions.yaml b/examples/mount_options/storageclass-wekafs-mountoptions.yaml index ccf984249..9fb058554 100644 --- a/examples/mount_options/storageclass-wekafs-mountoptions.yaml +++ b/examples/mount_options/storageclass-wekafs-mountoptions.yaml @@ -13,7 +13,7 @@ parameters: filesystemGroupName: default # comma-separated list of extra mount options to be passed to the mounter. NOTE: No validation whatsoever is performed on those - mountOptions: "readcache, noatime" + mountOptions: "readcache,noatime" # name of the secret that stores API credentials for a cluster # change the name of secret to match secret of a particular cluster (if you have several Weka clusters)