Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add csi-proxy project #362

Merged
merged 1 commit into from
Jul 23, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- [Kubernetes Cluster Controllers](kubernetes-cluster-controllers.md)
- [Snapshot Controller](snapshot-controller.md)
- [Snapshot Validation Webhook](snapshot-validation-webhook.md)
- [CSI Proxy](csi-proxy.md)
- [Sidecar Containers](sidecar-containers.md)
- [external-attacher](external-attacher.md)
- [external-provisioner](external-provisioner.md)
Expand Down Expand Up @@ -38,6 +39,7 @@
- [Volume Health Monitoring](volume-health-monitor.md)
- [Token Requests](token-requests.md)
- [FSGroup Support](support-fsgroup.md)
- [CSI Windows](csi-windows.md)
- [Deploying a CSI Driver on Kubernetes](deploying.md)
- [Example](example.md)
- [Driver Testing](testing-drivers.md)
Expand Down
46 changes: 46 additions & 0 deletions book/src/csi-proxy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# CSI Proxy

## Status and Releases

**Git Repository:** [https://github.com/kubernetes-csi/csi-proxy](https://github.com/kubernetes-csi/csi-proxy)

**Status:** V1 starting with v1.0.0

Status | Min K8s Version | Max K8s Version
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the other component pages, can you add a column for where the community binaries are located?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added, right now only in staging

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will update this after promote to release is available.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's update the entry here with v1.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

--|--|--
v0.1.0 | 1.18 | -
v0.2.0+ | 1.18 | -
v1.0.0+ | 1.18 | -

## Description

CSI Proxy is a binary that exposes a set of gRPC APIs around storage operations over named pipes in Windows. A container, such as CSI node plugins, can mount the named pipes depending on operations it wants to exercise on the host and invoke the APIs.

Each named pipe will support a specific version of an API (e.g. v1alpha1, v2beta1) that targets a specific area of storage (e.g. disk, volume, file, SMB, iSCSI). For example, `\\.\pipe\csi-proxy-filesystem-v1alpha1`, `\\.\pipe\csi-proxy-disk-v1beta1`. Any release of csi-proxy.exe binary will strive to maintain backward compatibility across as many prior stable versions of an API group as possible. Please see details in this [CSI Windows support KEP](https://github.com/kubernetes/enhancements/tree/master/keps/sig-windows/1122-windows-csi-support)

## Usage

Run csi-proxy.exe binary directly on a Windows node. The command line options are:

* `-kubelet-path`: This is the prefix path of the kubelet directory in the host file system (the default value is set to `C:\var\lib\kubelet`)

* `-windows-service`: Configure as a Windows Service
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these options also available in 1.0? The indenting makes it seem like these options are not available in 1.0 anymore. Maybe for simplicilty, let's remove the detailed entries for the old kubelet path arguments, and just mention in the description for kubelet-path that it replaces kubelet-pod-path and kubelet-csi-plugins-path in previous versions.


* `-log_file`: If non-empty, use this log file. (Note: must set `logtostdrr`=false if setting -log_file)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a flag from klog that uses underscores? (vs dashes for the other args)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.



Note that `-kubelet-pod-path` and `-kubelet-csi-plugins-path` were used in prior 1.0.0 versions, and they are now replaced by new parameter `-kubelet-path`

For detailed information (binary parameters, etc.), see the README of the relevant branch.

## Deployment

It the responsibility of the Kubernetes distribution or cluster admin to install csi-proxy. Directly run csi-proxy.exe binary or run it as a Windows Service on Kubernetes nodes.
For example,

```
$flags = "-windows-service -log_file=\etc\kubernetes\logs\csi-proxy.log -logtostderr=false"
sc.exe create csiproxy binPath= "${env:NODE_DIR}\csi-proxy.exe $flags"
sc.exe failure csiproxy reset= 0 actions= restart/10000
sc.exe start csiproxy
```
100 changes: 100 additions & 0 deletions book/src/csi-windows.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# CSI Windows Suppoort

## Status

Status | Min K8s Version | Min CSI proxy Version | Min Node Driver Registrar Version
--|--|--|--
GA | 1.19 | 1.0.0 | 1.3.0
Beta | 1.19 | 0.2.0 | 1.3.0
Alpha | 1.18 | 0.1.0 | 1.3.0


## Overview

CSI drivers (e.g. AzureDisk, GCE PD, etc.) are recommended to be deployed as containers. CSI driver’s node plugin typically runs on every worker node in the cluster (as a DaemonSet). Node plugin containers need to run with elevated privileges to perform storage related operations. However, Windows was not supporting privileged containers (Note: privileged containers a.k.a Host process is introduced as alpha feature in Kubernetes 1.22 very recently). To solve this problem, [CSI Proxy](https://github.com/kubernetes-csi/csi-proxy) is a binary that runs on the Windows host and executes a set of privileged storage operations on Windows nodes on behalf of containers in a CSI Node plugin daemonset. This enables multiple CSI Node plugins to execute privileged storage operations on Windows nodes without having to ship a custom privileged operation proxy.

Please note that CSI controller level operations/sidecars are not supported on Windows.

## How to use the CSI Proxy for Windows?
See how to install CSI Proxy in [csi-proxy.md#Deployment]

For CSI driver authors, import CSI proxy client under github.com/kubernetes-csi/csi-proxy/client. There are six client API groups including disk, filesystem, iscsi, smb, system, volume. See [link](https://github.com/kubernetes-csi/csi-proxy/tree/master/client/groups) for details.
As an example, please check how GCE PD Driver import disk, volume and filesystem client API groups [here](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/release-1.2/pkg/mount-manager/safe-mounter_windows.go#L28)

The Daemonset specification of a CSI node plugin for Windows can mount the desired named pipes from CSI Proxy based on the version of the API groups that the node-plugin needs to execute.


The following Daemonset YAML shows how to mount various API groups from CSI Proxy into a CSI Node plugin:

```
kind: DaemonSet
apiVersion: apps/v1
metadata:
name: csi-storage-node-win
spec:
selector:
matchLabels:
app: csi-driver-win
template:
metadata:
labels:
app: csi-driver-win
spec:
serviceAccountName: csi-node-sa
nodeSelector:
kubernetes.io/os: windows
containers:
- name: csi-driver-registrar
image: k8s.gcr.io/sig-storage/csi-node-driver-registrar
args:
- "--v=5"
- "--csi-address=unix://C:\\csi\\csi.sock"
- "--kubelet-registration-path=C:\\kubelet\\plugins\\plugin.csi\\csi.sock"
volumeMounts:
- name: plugin-dir
mountPath: C:\csi
- name: registration-dir
mountPath: C:\registration
- name: csi-driver
image: k8s.gcr.io/sig-storage/csi-driver:win-v1
args:
- "--v=5"
- "--endpoint=unix:/csi/csi.sock"
volumeMounts:
- name: kubelet-dir
mountPath: C:\var\lib\kubelet
- name: plugin-dir
mountPath: C:\csi
- name: csi-proxy-disk-pipe
mountPath: \\.\pipe\csi-proxy-disk-v1
- name: csi-proxy-volume-pipe
mountPath: \\.\pipe\csi-proxy-volume-v1
- name: csi-proxy-filesystem-pipe
mountPath: \\.\pipe\csi-proxy-filesystem-v1
volumes:
- name: csi-proxy-disk-pipe
hostPath:
path: \\.\pipe\csi-proxy-disk-v1
type: ""
- name: csi-proxy-volume-pipe
hostPath:
path: \\.\pipe\csi-proxy-volume-v1
type: ""
- name: csi-proxy-filesystem-pipe
hostPath:
path: \\.\pipe\csi-proxy-filesystem-v1
type: ""
- name: registration-dir
hostPath:
path: C:\var\lib\kubelet\plugins_registry\
type: Directory
- name: kubelet-dir
hostPath:
path: C:\var\lib\kubelet\
type: Directory
- name: plugin-dir
hostPath:
path: C:\var\lib\kubelet\plugins\csi.org.io\
type: DirectoryOrCreate
```