-
Notifications
You must be signed in to change notification settings - Fork 250
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add csi-proxy project #362
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
# CSI Proxy | ||
|
||
## Status and Releases | ||
|
||
**Git Repository:** [https://github.com/kubernetes-csi/csi-proxy](https://github.com/kubernetes-csi/csi-proxy) | ||
|
||
**Status:** V1 starting with v1.0.0 | ||
|
||
Status | Min K8s Version | Max K8s Version | ||
--|--|-- | ||
v0.1.0 | 1.18 | - | ||
v0.2.0+ | 1.18 | - | ||
v1.0.0+ | 1.18 | - | ||
|
||
## Description | ||
|
||
CSI Proxy is a binary that exposes a set of gRPC APIs around storage operations over named pipes in Windows. A container, such as CSI node plugins, can mount the named pipes depending on operations it wants to exercise on the host and invoke the APIs. | ||
|
||
Each named pipe will support a specific version of an API (e.g. v1alpha1, v2beta1) that targets a specific area of storage (e.g. disk, volume, file, SMB, iSCSI). For example, `\\.\pipe\csi-proxy-filesystem-v1alpha1`, `\\.\pipe\csi-proxy-disk-v1beta1`. Any release of csi-proxy.exe binary will strive to maintain backward compatibility across as many prior stable versions of an API group as possible. Please see details in this [CSI Windows support KEP](https://github.com/kubernetes/enhancements/tree/master/keps/sig-windows/1122-windows-csi-support) | ||
|
||
## Usage | ||
|
||
Run csi-proxy.exe binary directly on a Windows node. The command line options are: | ||
|
||
* `-kubelet-path`: This is the prefix path of the kubelet directory in the host file system (the default value is set to `C:\var\lib\kubelet`) | ||
|
||
* `-windows-service`: Configure as a Windows Service | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are these options also available in 1.0? The indenting makes it seem like these options are not available in 1.0 anymore. Maybe for simplicilty, let's remove the detailed entries for the old kubelet path arguments, and just mention in the description for |
||
|
||
* `-log_file`: If non-empty, use this log file. (Note: must set `logtostdrr`=false if setting -log_file) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this a flag from klog that uses underscores? (vs dashes for the other args) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. https://github.com/kubernetes-csi/csi-proxy/blob/master/vendor/k8s.io/klog/v2/klog.go#L427 This is from klog option, not csi-proxy actually |
||
|
||
|
||
Note that `-kubelet-pod-path` and `-kubelet-csi-plugins-path` were used in prior 1.0.0 versions, and they are now replaced by new parameter `-kubelet-path` | ||
|
||
For detailed information (binary parameters, etc.), see the README of the relevant branch. | ||
|
||
## Deployment | ||
|
||
It the responsibility of the Kubernetes distribution or cluster admin to install csi-proxy. Directly run csi-proxy.exe binary or run it as a Windows Service on Kubernetes nodes. | ||
For example, | ||
|
||
``` | ||
$flags = "-windows-service -log_file=\etc\kubernetes\logs\csi-proxy.log -logtostderr=false" | ||
sc.exe create csiproxy binPath= "${env:NODE_DIR}\csi-proxy.exe $flags" | ||
sc.exe failure csiproxy reset= 0 actions= restart/10000 | ||
sc.exe start csiproxy | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,100 @@ | ||
# CSI Windows Suppoort | ||
|
||
## Status | ||
|
||
Status | Min K8s Version | Min CSI proxy Version | Min Node Driver Registrar Version | ||
--|--|--|-- | ||
GA | 1.19 | 1.0.0 | 1.3.0 | ||
Beta | 1.19 | 0.2.0 | 1.3.0 | ||
Alpha | 1.18 | 0.1.0 | 1.3.0 | ||
|
||
|
||
## Overview | ||
|
||
CSI drivers (e.g. AzureDisk, GCE PD, etc.) are recommended to be deployed as containers. CSI driver’s node plugin typically runs on every worker node in the cluster (as a DaemonSet). Node plugin containers need to run with elevated privileges to perform storage related operations. However, Windows was not supporting privileged containers (Note: privileged containers a.k.a Host process is introduced as alpha feature in Kubernetes 1.22 very recently). To solve this problem, [CSI Proxy](https://github.com/kubernetes-csi/csi-proxy) is a binary that runs on the Windows host and executes a set of privileged storage operations on Windows nodes on behalf of containers in a CSI Node plugin daemonset. This enables multiple CSI Node plugins to execute privileged storage operations on Windows nodes without having to ship a custom privileged operation proxy. | ||
|
||
Please note that CSI controller level operations/sidecars are not supported on Windows. | ||
|
||
## How to use the CSI Proxy for Windows? | ||
See how to install CSI Proxy in [csi-proxy.md#Deployment] | ||
|
||
For CSI driver authors, import CSI proxy client under github.com/kubernetes-csi/csi-proxy/client. There are six client API groups including disk, filesystem, iscsi, smb, system, volume. See [link](https://github.com/kubernetes-csi/csi-proxy/tree/master/client/groups) for details. | ||
As an example, please check how GCE PD Driver import disk, volume and filesystem client API groups [here](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/blob/release-1.2/pkg/mount-manager/safe-mounter_windows.go#L28) | ||
|
||
The Daemonset specification of a CSI node plugin for Windows can mount the desired named pipes from CSI Proxy based on the version of the API groups that the node-plugin needs to execute. | ||
|
||
|
||
The following Daemonset YAML shows how to mount various API groups from CSI Proxy into a CSI Node plugin: | ||
|
||
``` | ||
kind: DaemonSet | ||
apiVersion: apps/v1 | ||
metadata: | ||
name: csi-storage-node-win | ||
spec: | ||
selector: | ||
matchLabels: | ||
app: csi-driver-win | ||
template: | ||
metadata: | ||
labels: | ||
app: csi-driver-win | ||
spec: | ||
serviceAccountName: csi-node-sa | ||
nodeSelector: | ||
kubernetes.io/os: windows | ||
containers: | ||
- name: csi-driver-registrar | ||
image: k8s.gcr.io/sig-storage/csi-node-driver-registrar | ||
args: | ||
- "--v=5" | ||
- "--csi-address=unix://C:\\csi\\csi.sock" | ||
- "--kubelet-registration-path=C:\\kubelet\\plugins\\plugin.csi\\csi.sock" | ||
volumeMounts: | ||
- name: plugin-dir | ||
mountPath: C:\csi | ||
- name: registration-dir | ||
mountPath: C:\registration | ||
- name: csi-driver | ||
image: k8s.gcr.io/sig-storage/csi-driver:win-v1 | ||
args: | ||
- "--v=5" | ||
- "--endpoint=unix:/csi/csi.sock" | ||
volumeMounts: | ||
- name: kubelet-dir | ||
mountPath: C:\var\lib\kubelet | ||
- name: plugin-dir | ||
mountPath: C:\csi | ||
- name: csi-proxy-disk-pipe | ||
mountPath: \\.\pipe\csi-proxy-disk-v1 | ||
- name: csi-proxy-volume-pipe | ||
mountPath: \\.\pipe\csi-proxy-volume-v1 | ||
- name: csi-proxy-filesystem-pipe | ||
mountPath: \\.\pipe\csi-proxy-filesystem-v1 | ||
volumes: | ||
- name: csi-proxy-disk-pipe | ||
hostPath: | ||
path: \\.\pipe\csi-proxy-disk-v1 | ||
type: "" | ||
- name: csi-proxy-volume-pipe | ||
hostPath: | ||
path: \\.\pipe\csi-proxy-volume-v1 | ||
type: "" | ||
- name: csi-proxy-filesystem-pipe | ||
hostPath: | ||
path: \\.\pipe\csi-proxy-filesystem-v1 | ||
type: "" | ||
- name: registration-dir | ||
hostPath: | ||
path: C:\var\lib\kubelet\plugins_registry\ | ||
type: Directory | ||
- name: kubelet-dir | ||
hostPath: | ||
path: C:\var\lib\kubelet\ | ||
type: Directory | ||
- name: plugin-dir | ||
hostPath: | ||
path: C:\var\lib\kubelet\plugins\csi.org.io\ | ||
type: DirectoryOrCreate | ||
``` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similar to the other component pages, can you add a column for where the community binaries are located?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added, right now only in staging
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will update this after promote to release is available.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's update the entry here with v1.0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated