Add kubernetes-cluster receiver (#175)

### Description Add `k8s_cluster` receiver. This receiver monitors resources in a cluster and collects metrics and metadata to correlate between resources. This receiver watches for changes using the K8s API. ##### Key Data Structures - `kubernetesReceiver`: Sends metrics along the pipeline - `resourceWatcher` : Handles events from the K8s API, collecting metrics and metadata from the cluster. This struct uses a `informers.SharedInformerFactory` to watch for events. - `dataCollector` : Handles collection of data from Kubernetes resources. The collector has a `metricsStore` to keep track of latest metrics representing the cluster state and a `metadataStore` (a wrapper around `cache.Store`) to track latest metadata from the cluster. ##### Workflow - **resourceWatcher setup** - setup SharedInformerFactory, add event handler to informers. The following methods:`onAdd`, `onUpdate` and `onDelete` on the `resourceWatcher` handle resource creations, deletions and updates. - **Event Handling** - _Add/Update_: On receiving an event corresponding to a resource creation or update, the latest metrics are collected by `syncMetrics` method on `dataCollector`. The collected metrics are cached in `metricsStore`. Methods responsible for collecting data from each supported Kubernetes resource type are prefixed with `getMetricsFor`. For example, `getMetricsForPod` collects metrics from Pod. - _Delete_: On deletion of resources, the cached entry will be removed from `metricsStore`. - Note that only metric collection is turned on right now. The metadata collection code is currently inactive (It is controlled by the `collectMedata` field). - **Metric Syncing**: Metrics from the `metricStore` are send along the pipeline once every `collection_interval` seconds. - **Metadata Syncing**: TODO (The metadata collection code is inactive) ### Testing - Unit tests for each resource type - Integration test for receiver - Manual testing with SignalFx exporter ### Documentation https://github.com/signalfx/opentelemetry-collector-contrib/blob/k8s-cluster/receiver/kubernetesclusterreceiver/README.md
open-telemetry · Apr 27, 2020 · 9ee65b1 · 9ee65b1
1 parent e3017c7
commit 9ee65b1
Show file tree

Hide file tree

Showing 52 changed files with 5,839 additions and 92 deletions.
diff --git a/cmd/otelcontribcol/components.go b/cmd/otelcontribcol/components.go
@@ -34,6 +34,7 @@ import (
 	"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/carbonreceiver"
 	"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/collectdreceiver"
 	"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jaegerlegacyreceiver"
+	"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver"
 	"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/redisreceiver"
 	"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/sapmreceiver"
 	"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/signalfxreceiver"
@@ -57,6 +58,7 @@ func components() (config.Factories, error) {
 		&wavefrontreceiver.Factory{},
 		&jaegerlegacyreceiver.Factory{},
 		&redisreceiver.Factory{},
+		&k8sclusterreceiver.Factory{},
 	}
 	for _, rcv := range factories.Receivers {
 		receivers = append(receivers, rcv)

diff --git a/go.mod b/go.mod
@@ -20,6 +20,7 @@ require (
 	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/carbonreceiver v0.0.0
 	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/collectdreceiver v0.0.0
 	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jaegerlegacyreceiver v0.0.0
+	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver v0.0.0
 	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/redisreceiver v0.0.0
 	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/sapmreceiver v0.0.0
 	github.com/open-telemetry/opentelemetry-collector-contrib/receiver/signalfxreceiver v0.0.0
@@ -71,6 +72,8 @@ replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/recei
 
 replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/sapmreceiver => ./receiver/sapmreceiver
 
+replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver => ./receiver/k8sclusterreceiver
+
 replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/signalfxreceiver => ./receiver/signalfxreceiver
 
 replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/wavefrontreceiver => ./receiver/wavefrontreceiver

diff --git a/go.sum b/go.sum
diff --git a/receiver/k8sclusterreceiver/Makefile b/receiver/k8sclusterreceiver/Makefile
@@ -0,0 +1 @@
+include ../../Makefile.Common
diff --git a/receiver/k8sclusterreceiver/README.md b/receiver/k8sclusterreceiver/README.md
@@ -0,0 +1,229 @@
+# Kubernetes Cluster Receiver
+
+**Status: beta**
+
+### Overview
+
+The Kubernetes Cluster receiver collects cluster-level metrics from the Kubernetes 
+API server. It uses the K8s API to listen for updates. A single instance of this 
+receiver can be used to monitor a cluster.
+
+Currently this receiver supports authentication via service accounts only. See [example](#example) 
+for more information.
+
+### Config
+
+#### collection_interval
+
+This receiver continuously watches for events using K8s API. However, the metrics
+collected are emitted only once every collection interval. `collection_interval` will
+determine the frequency at which metrics are emitted by this receiver.
+
+default: `10s`
+
+#### node_conditions_to_report
+
+An array of node conditions this receiver should report. See [here](https://kubernetes.io/docs/concepts/architecture/nodes/#condition)
+for list of node conditions. The receiver will emit one metric per entry in the
+array.
+
+```yaml
+...
+k8s_cluster:
+  node_conditions_to_report:
+    - Ready
+    - MemoryPressure
+...
+```
+
+For example, with the above config the receiver will emit two metrics
+`kubernetes/node/condition_ready` and `kubernetes/node/condition_memory_pressure`, one
+for each condition in the config. The value will be `1` if the `ConditionStatus` for the
+corresponding `Condition` is `True`, `0` if it is `False` and -1 if it is `Unknown`.
+
+
+default: `[Ready]`
+
+### Example
+
+Here is an example deployment of the collector that sets up this receiver along with 
+the [SignalFx Exporter](../../exporter/signalfxexporter/README.md).
+
+Follow the below sections to setup various Kubernetes resources required for the deployment.
+
+#### Config
+
+Create a ConfigMap with the config for `otelcontribcol`. Replace `SIGNALFX_TOKEN` and `SIGNAL_INGEST_URL` 
+with valid values.
+
+```bash
+cat <<EOF | kubectl apply -f -
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: otelcontribcol
+  labels:
+    app: otelcontribcol
+data:
+  config.yaml: |
+    receivers:
+      k8s_cluster:
+        collection_interval: 10s
+        #k8s_config: k8s-config
+    exporters:
+      signalfx:
+        access_token: <SIGNALFX_TOKEN>
+        url: <SIGNALFX_INGEST_URL>
+
+    service:
+      pipelines:
+        metrics:
+          receivers: [k8s_cluster]
+          exporters: [signalfx]
+EOF
+```
+
+#### Service Account
+
+Create a service account that the collector should use.
+
+```bash
+<<EOF | kubectl apply -f -
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  labels:
+    app: otelcontribcol
+  name: otelcontribcol
+EOF
+```
+
+#### RBAC
+
+Use the below commands to create a `ClusterRole` with required permissions and a 
+`ClusterRoleBinding` to grant the role to the service account created above.
+
+```bash
+<<EOF | kubectl apply -f -
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRole
+metadata:
+  name: otelcontribcol
+  labels:
+    app: otelcontribcol
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - events
+  - namespaces
+  - namespaces/status
+  - nodes
+  - nodes/spec
+  - pods
+  - pods/status
+  - replicationcontrollers
+  - replicationcontrollers/status
+  - resourcequotas
+  - services
+  verbs:
+  - get
+  - list
+  - watch
+- apiGroups:
+  - apps
+  resources:
+  - daemonsets
+  - deployments
+  - replicasets
+  - statefulsets
+  verbs:
+  - get
+  - list
+  - watch
+- apiGroups:
+  - extensions
+  resources:
+  - daemonsets
+  - deployments
+  - replicasets
+  verbs:
+  - get
+  - list
+  - watch
+- apiGroups:
+  - batch
+  resources:
+  - jobs
+  - cronjobs
+  verbs:
+  - get
+  - list
+  - watch
+- apiGroups:
+    - autoscaling
+  resources:
+    - horizontalpodautoscalers
+  verbs:
+    - get
+    - list
+    - watch
+EOF
+```
+
+```bash
+<<EOF | kubectl apply -f -
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: otelcontribcol
+  labels:
+    app: otelcontribcol
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: otelcontribcol
+subjects:
+- kind: ServiceAccount
+  name: otelcontribcol
+  namespace: default
+EOF
+```
+
+#### Deployment
+
+Create a [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) to deploy the collector.
+
+```bash
+<<EOF | kubectl apply -f -
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: otelcontribcol
+  labels:
+    app: otelcontribcol
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: otelcontribcol
+  template:
+    metadata:
+      labels:
+        app: otelcontribcol
+    spec:
+      serviceAccountName: otelcontribcol
+      containers:
+      - name: otelcontribcol
+        image: otelcontribcol:latest # specify image
+        args: ["--config", "/etc/config/config.yaml"]
+        volumeMounts:
+        - name: config
+          mountPath: /etc/config
+        imagePullPolicy: IfNotPresent
+      volumes:
+        - name: config
+          configMap:
+            name: otelcontribcol
+EOF
+```