Add multitenant example for Tempo (#11)

* Add multitenant example #10 Signed-off-by: Israel Blancas <[email protected]> * Add missing namespace Signed-off-by: Israel Blancas <[email protected]> * Improve example documentation * Remove usage of minio. Add resources to allow the UI access the traces from the different tenants * Apply feedback requested in PR * Fix typos * Add feedback from code review --------- Signed-off-by: Israel Blancas <[email protected]>
os-observability · May 26, 2023 · 0a97b7c · 0a97b7c
1 parent 827e7f9
commit 0a97b7c
Show file tree

Hide file tree

Showing 2 changed files with 164 additions and 0 deletions.
diff --git a/tempo/auth/README.md b/tempo/auth/README.md
@@ -0,0 +1,106 @@
+# Tempo authentication
+
+This is an example about how to deploy a Tempo instance using the Tempo Operator. It receives traces from two tenants (`prod` and `dev`).
+
+
+The Tempo deployment accepts traces from two tenants: `dev` and `prod`. The way to identify the tenants is through the `X-Scope-OrgID` OTLP header. When the `X-Scope-OrgID` header is set to `dev` in the trace, the tenant is `dev`. When the `X-Scope-OrgID` header is set to `prod` in the trace, the tenant is `prod`.
+
+It also enables the creation of the Jaeger UI `.spec.template.queryFrontend.jaegerQuery.enabled` and the gateway (`.spec.template.gateway.enabled`). You can get the URL to the UI for a given tenant with the following command:
+
+```sh
+echo https://$(oc get routes tempo-authentication-example-gateway -o jsonpath='{.spec.host}')/<tenant name>
+```
+
+For instance, for the `dev` tenant, it would be:
+```sh
+echo https://$(oc get routes tempo-authentication-example-gateway -o jsonpath='{.spec.host}')/dev
+```
+
+The tenant needs permissions to the `tempo.grafana.com` API Group to write traces. You can follow this example to know
+how to provide write permissions to the `tenant-sa` `ServiceAccount` to write traces for the `dev` tenant:
+```yaml
+# Create the Service Account
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: tenant-sa
+  namespace: other-namespace
+---
+# ClusterRole needed to grant permissions to the service account to write traces
+# for the given tenant
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: tempostack-traces-write
+rules:
+  - apiGroups:
+      - 'tempo.grafana.com'
+    # Tenant name set in X-Scope-OrgID
+    resources:
+      - dev
+    resourceNames:
+      - traces
+    verbs:
+      - 'create'
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: tempostack-traces
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: tempostack-traces-write
+subjects:
+  - kind: ServiceAccount
+    name: tenant-sa
+    namespace: other-namespace
+```
+Now, you can use the `tenant-sa` `ServiceAccount` in your application or `OpenTelemetry Collector` instance to write traces for the `dev` tenant.
+
+To read the traces, you also need to create a `ClusterRoleBinding` to give `get` permissions to the `traces` resource from the `tempo.grafana.com` API group. In this example, we create a `ClusterRoleBinding` that gives read access to the traces from the `dev` and `prod` tenants to the authenticated users:
+```yaml
+# Allow the Jaeger UI to retrieve the data from the dev and prod tenants
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: tempostack-traces-reader
+rules:
+  - apiGroups:
+      - 'tempo.grafana.com'
+    resources:
+      - dev
+      - prod
+    resourceNames:
+      - traces
+    verbs:
+      - 'get'
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: tempostack-traces-reader
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: tempostack-traces-reader
+subjects:
+  - kind: Group
+    apiGroup: rbac.authorization.k8s.io
+    name: system:authenticated
+```
+
+## How to run
+1. Create an Object storage instance using [OpenShift Data Foundation](https://access.redhat.com/documentation/en-us/red_hat_openshift_data_foundation/).
+1. Create an Object Storage secret with keys as follows:
+    ```console
+    kubectl create secret generic object-storage \
+      --from-literal=bucket="<BUCKET_NAME>" \
+      --from-literal=endpoint="https://s3.openshift-storage.svc" \
+      --from-literal=access_key_id="<ACCESS_KEY_ID>" \
+      --from-literal=access_key_secret="<ACCESS_KEY_SECRET>"
+    ```
+1. Deploy the Tempo instance in the `tempo-example` OpenShift Project:
+    ```sh
+    kubectl create -f tempo.yaml
+    ```
diff --git a/tempo/auth/tempo.yaml b/tempo/auth/tempo.yaml
@@ -0,0 +1,58 @@
+apiVersion: project.openshift.io/v1
+kind: Project
+metadata:
+  name: tempo-example
+---
+# Allow the Jaeger UI to retrieve the data from the dev and prod tenants
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: tempostack-traces-reader
+rules:
+  - apiGroups:
+      - 'tempo.grafana.com'
+    resources:
+      - dev
+      - prod
+    resourceNames:
+      - traces
+    verbs:
+      - 'get'
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: tempostack-traces-reader
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: tempostack-traces-reader
+subjects:
+  - kind: Group
+    apiGroup: rbac.authorization.k8s.io
+    name: system:authenticated
+---
+apiVersion: tempo.grafana.com/v1alpha1
+kind:  TempoStack
+metadata:
+  name: authentication-example
+  namespace: tempo-example
+spec:
+  storage:
+    secret:
+      name: object-storage
+      type: s3
+  storageSize: 1Gi
+  tenants:
+    mode: openshift
+    authentication:
+      - tenantName: dev
+        tenantId: "1610b0c3-c509-4592-a256-a1871353dbfa"
+      - tenantName: prod
+        tenantId: "1610b0c3-c509-4592-a256-a1871353dbfb"
+  template:
+    gateway:
+      enabled: true
+    queryFrontend:
+      jaegerQuery:
+        enabled: true