From 93bd708997a28c96d247a23cbb1a6457953f9868 Mon Sep 17 00:00:00 2001 From: Jing Xu Date: Wed, 9 Sep 2020 22:44:35 -0700 Subject: [PATCH] Add csi-proxy project Add csi-proxy project in docs --- book/src/SUMMARY.md | 2 + book/src/csi-proxy.md | 46 ++++++++++++++++++ book/src/csi-windows.md | 100 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 148 insertions(+) create mode 100644 book/src/csi-proxy.md create mode 100644 book/src/csi-windows.md diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 3abbfd9a..02c705e3 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -7,6 +7,7 @@ - [Kubernetes Cluster Controllers](kubernetes-cluster-controllers.md) - [Snapshot Controller](snapshot-controller.md) - [Snapshot Validation Webhook](snapshot-validation-webhook.md) + - [CSI Proxy](csi-proxy.md) - [Sidecar Containers](sidecar-containers.md) - [external-attacher](external-attacher.md) - [external-provisioner](external-provisioner.md) @@ -38,6 +39,7 @@ - [Volume Health Monitoring](volume-health-monitor.md) - [Token Requests](token-requests.md) - [FSGroup Support](support-fsgroup.md) + - [CSI Windows](csi-windows.md) - [Deploying a CSI Driver on Kubernetes](deploying.md) - [Example](example.md) - [Driver Testing](testing-drivers.md) diff --git a/book/src/csi-proxy.md b/book/src/csi-proxy.md new file mode 100644 index 00000000..3379abe0 --- /dev/null +++ b/book/src/csi-proxy.md @@ -0,0 +1,46 @@ +# CSI Proxy + +## Status and Releases + +**Git Repository:** [https://github.com/kubernetes-csi/csi-proxy](https://github.com/kubernetes-csi/csi-proxy) + +**Status:** V1 starting with v1.0.0 + +Status | Min K8s Version | Max K8s Version +--|--|-- +v0.1.0 | 1.18 | - +v0.2.0+ | 1.18 | - +v1.0.0+ | 1.18 | - + +## Description + +CSI Proxy is a binary that exposes a set of gRPC APIs around storage operations over named pipes in Windows. A container, such as CSI node plugins, can mount the named pipes depending on operations it wants to exercise on the host and invoke the APIs. + +Each named pipe will support a specific version of an API (e.g. v1alpha1, v2beta1) that targets a specific area of storage (e.g. disk, volume, file, SMB, iSCSI). For example, `\\.\pipe\csi-proxy-filesystem-v1alpha1`, `\\.\pipe\csi-proxy-disk-v1beta1`. Any release of csi-proxy.exe binary will strive to maintain backward compatibility across as many prior stable versions of an API group as possible. Please see details in this [CSI Windows support KEP](https://github.com/kubernetes/enhancements/tree/master/keps/sig-windows/1122-windows-csi-support) + +## Usage + +Run csi-proxy.exe binary directly on a Windows node. The command line options are: + +* `-kubelet-path`: This is the prefix path of the kubelet directory in the host file system (the default value is set to `C:\var\lib\kubelet`) + +* `-windows-service`: Configure as a Windows Service + +* `-log_file`: If non-empty, use this log file. (Note: must set `logtostdrr`=false if setting -log_file) + + +Note that `-kubelet-pod-path` and `-kubelet-csi-plugins-path` were used in prior 1.0.0 versions, and they are now replaced by new parameter `-kubelet-path` + +For detailed information (binary parameters, etc.), see the README of the relevant branch. + +## Deployment + +It the responsibility of the Kubernetes distribution or cluster admin to install csi-proxy. Directly run csi-proxy.exe binary or run it as a Windows Service on Kubernetes nodes. +For example, + +``` + $flags = "-windows-service -log_file=\etc\kubernetes\logs\csi-proxy.log -logtostderr=false" + sc.exe create csiproxy binPath= "${env:NODE_DIR}\csi-proxy.exe $flags" + sc.exe failure csiproxy reset= 0 actions= restart/10000 + sc.exe start csiproxy +``` diff --git a/book/src/csi-windows.md b/book/src/csi-windows.md new file mode 100644 index 00000000..e34fa7fd --- /dev/null +++ b/book/src/csi-windows.md @@ -0,0 +1,100 @@ +# CSI Windows Suppoort + +## Status + +Status | Min K8s Version | Min CSI proxy Version | Min Node Driver Registrar Version +--|--|--|-- +GA | 1.19 | 1.0.0 | 1.3.0 +Beta | 1.19 | 0.2.0 | 1.3.0 +Alpha | 1.18 | 0.1.0 | 1.3.0 + + +## Overview + +CSI drivers (e.g. AzureDisk, GCE PD, etc.) are recommended to be deployed as containers. CSI driver’s node plugin typically runs on every worker node in the cluster (as a DaemonSet). Node plugin containers need to run with elevated privileges to perform storage related operations. However, Windows was not supporting privileged containers (Note: privileged containers a.k.a Host process is introduced as alpha feature in Kubernetes 1.22 very recently). To solve this problem, [CSI Proxy](https://github.com/kubernetes-csi/csi-proxy) is a binary that runs on the Windows host and executes a set of privileged storage operations on Windows nodes on behalf of containers in a CSI Node plugin daemonset. This enables multiple CSI Node plugins to execute privileged storage operations on Windows nodes without having to ship a custom privileged operation proxy. + +Please note that CSI controller level operations/sidecars are not supported on Windows. + +## How to use the CSI Proxy for Windows? +See how to install CSI Proxy in [csi-proxy.md#Deployment] + +For CSI driver authors, import CSI proxy client under github.com/kubernetes-csi/csi-proxy/client. There are six client API groups including disk, filesystem, iscsi, smb, system, volume. See [link](https://github.com/kubernetes-csi/csi-proxy/tree/master/client/groups) for details. +As an example, please check how GCE PD Driver import disk, volume and filesystem client API groups [here](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/release-1.2/pkg/mount-manager/safe-mounter_windows.go#L28) + +The Daemonset specification of a CSI node plugin for Windows can mount the desired named pipes from CSI Proxy based on the version of the API groups that the node-plugin needs to execute. + + +The following Daemonset YAML shows how to mount various API groups from CSI Proxy into a CSI Node plugin: + +``` +kind: DaemonSet +apiVersion: apps/v1 +metadata: + name: csi-storage-node-win +spec: + selector: + matchLabels: + app: csi-driver-win + template: + metadata: + labels: + app: csi-driver-win + spec: + serviceAccountName: csi-node-sa + nodeSelector: + kubernetes.io/os: windows + containers: + - name: csi-driver-registrar + image: k8s.gcr.io/sig-storage/csi-node-driver-registrar + args: + - "--v=5" + - "--csi-address=unix://C:\\csi\\csi.sock" + - "--kubelet-registration-path=C:\\kubelet\\plugins\\plugin.csi\\csi.sock" + volumeMounts: + - name: plugin-dir + mountPath: C:\csi + - name: registration-dir + mountPath: C:\registration + - name: csi-driver + image: k8s.gcr.io/sig-storage/csi-driver:win-v1 + args: + - "--v=5" + - "--endpoint=unix:/csi/csi.sock" + volumeMounts: + - name: kubelet-dir + mountPath: C:\var\lib\kubelet + - name: plugin-dir + mountPath: C:\csi + - name: csi-proxy-disk-pipe + mountPath: \\.\pipe\csi-proxy-disk-v1 + - name: csi-proxy-volume-pipe + mountPath: \\.\pipe\csi-proxy-volume-v1 + - name: csi-proxy-filesystem-pipe + mountPath: \\.\pipe\csi-proxy-filesystem-v1 + volumes: + - name: csi-proxy-disk-pipe + hostPath: + path: \\.\pipe\csi-proxy-disk-v1 + type: "" + - name: csi-proxy-volume-pipe + hostPath: + path: \\.\pipe\csi-proxy-volume-v1 + type: "" + - name: csi-proxy-filesystem-pipe + hostPath: + path: \\.\pipe\csi-proxy-filesystem-v1 + type: "" + - name: registration-dir + hostPath: + path: C:\var\lib\kubelet\plugins_registry\ + type: Directory + - name: kubelet-dir + hostPath: + path: C:\var\lib\kubelet\ + type: Directory + - name: plugin-dir + hostPath: + path: C:\var\lib\kubelet\plugins\csi.org.io\ + type: DirectoryOrCreate +``` +