diff --git a/deployment/pcm/README.md b/deployment/pcm/README.md index 20812229..a4a193ed 100644 --- a/deployment/pcm/README.md +++ b/deployment/pcm/README.md @@ -148,6 +148,9 @@ helm install prometheus prometheus-community/kube-prometheus-stack --set prometh kubectl get sts prometheus-prometheus-kube-prometheus-prometheus ``` +Note: `podMonitorSelectorNilUsesHelmValues` is disabled (set to false) so Prometheus operator will be able to handle PCM podMonitor deployed without extra `podMonitorLabels` or otherwise pcm need to be deployed like this: +`helm install pcm . --set podMonitor=true --set podMonitorLabels.release=prometheus` (assuming Prometheus operator was deployed as "prometheus") + #### 5) Deploy PCM helm chart ``` @@ -217,72 +220,14 @@ helm install pcm-metal . -f values-metal.yaml #### Direct method as non-privileged container (not recommended) -**TODO**: TO BE MOVED TO EXTERNAL FILE/SECTION - -**Note** PCM requires access to /dev/cpu device in read writer mode (MSR access) but it is no possible currently to mount devices in Kubernetes pods/containers in vanila Kubernetes. Please read this isses for more information https://github.com/kubernetes/kubernetes/issues/5607. - -##### a) Device injection using 3rd party device-plugin - - -TO run PCM with as non privileged pod, we can third party devices plugins e.g.: - -- https://github.com/smarter-project/smarter-device-manager -- https://github.com/squat/generic-device-plugin -- https://github.com/everpeace/k8s-host-device-plugin - -**Warning** This plugins were NOT audited for security concerns, **use it at your own risk**. - -Below is example how to pass /dev/cpu and /dev/mem using smarter-device-manager in kind based Kubernetes test cluster. - -``` -# Label node to deploy device plugin on that node -kubectl label node kind-control-plane smarter-device-manager=enabled - -# Install "smarter-device-manager" device plugin with only /dev/cpu and /dev/mem devices enabled: -git clone https://github.com/smarter-project/smarter-device-manager -helm install smarter-device-plugin --create-namespace --namespace smarter-device-plugin smarter-device-manager/charts/smarter-device-manager --set 'config[0].devicematch=^cpu$' --set 'config[0].nummaxdevices=1' --set 'config[1].devicematch=^mem$' --set 'config[1].nummaxdevices=1' - -# Check that cpu and mem devices are available - should return "1" -kubectl get node kind-control-plane -o json | jq .status.capacity +**Note** PCM requires access to /dev/cpu device in read-write mode (MSR access) but it is no possible currently to mount devices in Kubernetes pods/containers in vanilla Kubernetes for unprivileged containers. Please find more about this limitation https://github.com/kubernetes/kubernetes/issues/5607. -# Install pcm helm chart in unprivileged mode with extraResources for cpu and memory devices. -helm install pcm . --set privileged=false -f values-direct.yaml -f values-smarter-devices-cpu-mem.yaml -``` +To expose necessary devices to pcm-sensor-server, one can use: -##### b) Device injection using NRI plugin device-injection +a) Kubernetes device plugin (using Kubernetes [CDI](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/) interface), +b) containerd plugin (using [NRI](https://github.com/containerd/nri/) interface), -**TODO**: **Warning** This is work in progress, because it is needed to manually specific all /dev/cpu/XX/msr devices, which is unpractical in production (TO BE MOVED TO EXTERNAL FILE). - -``` -git clone https://github.com/containerd/nri/ -(cd nri/plugins/device-injector/ && go build ) -docker cp kind-control-plane:/etc/containerd/config.toml config.toml - -cat >>config.toml <>config.toml <