-
Notifications
You must be signed in to change notification settings - Fork 452
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add autoinstrumentation of NodeJS #507
Changes from 19 commits
ba0bede
8a3770a
c54a5bc
ae74544
3c5c95e
21d5433
9b7a81e
b0239d6
7cc5182
fb23588
a302e31
1bbe523
293a8e0
b02d1d9
0b72fa3
50785ae
cff5521
d063917
d51b0aa
b4b8cbb
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -155,27 +155,16 @@ EOF | |
|
||
### OpenTelemetry auto-instrumentation injection | ||
|
||
The operator can inject and configure OpenTelemetry auto-instrumentation libraries. At this moment, the operator can inject only OpenTelemetry [Java auto-instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation). | ||
The operator can inject and configure OpenTelemetry auto-instrumentation libraries. Currently Java and NodeJS are supported. | ||
|
||
The injection of the Java agent can be enabled by adding an annotation to the namespace, so that all pods within that namespace will get the instrumentation, or by adding the annotation to individual PodSpec objects, available as part of Deployment, Statefulset, and other resources. | ||
|
||
```bash | ||
instrumentation.opentelemetry.io/inject-java: "true" | ||
``` | ||
|
||
The value can be | ||
* `"false"` - do not inject | ||
* `"true"` - inject and `Instrumentation` resource from the namespace. | ||
* `"java-instrumentation"` - name of `Instrumentation` CR instance. | ||
|
||
In addition to the annotation, the following `CR` has to be created. The `Instrumentation` resource provides configuration for OpenTelemetry SDK and auto-instrumentation. | ||
To use auto-instrumentation, configure an `Instrumentation` resource with the configuration for the SDK and instrumentation. | ||
|
||
```yaml | ||
kubectl apply -f - <<EOF | ||
apiVersion: opentelemetry.io/v1alpha1 | ||
kind: Instrumentation | ||
metadata: | ||
name: java-instrumentation | ||
name: my-instrumentation | ||
spec: | ||
exporter: | ||
endpoint: http://otel-collector:4317 | ||
|
@@ -188,12 +177,42 @@ spec: | |
argument: "0.25" | ||
java: | ||
image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-java:latest # <1> | ||
EOF | ||
nodejs: | ||
image: ghcr.io/open-telemetry/opentelemetry-operator/autoinstrumentation-nodejs:latest | ||
EOF | ||
``` | ||
|
||
The above CR can be queried by `kubectl get otelinst`. | ||
|
||
1. Container image with [OpenTelemetry Java auto-instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation). The image must contain the Java agent JAR `/javaagent.jar`, and the operator will copy it to a shared volume mounted to the application container. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's delete this and consider it as impl detail for now |
||
|
||
The above CR can be queried by `kubectl get otelinst`. | ||
#### Java | ||
|
||
The operator can inject OpenTelemetry [Java auto-instrumentation](https://github.com/open-telemetry/opentelemetry-java-instrumentation). | ||
The injection of the Java agent can be enabled by adding an annotation to the namespace, so that all pods within that namespace will get the instrumentation, or by adding the annotation to individual PodSpec objects, available as part of Deployment, Statefulset, and other resources. | ||
|
||
```bash | ||
instrumentation.opentelemetry.io/inject-java: "true" | ||
``` | ||
|
||
The value can be | ||
* `"false"` - do not inject | ||
* `"true"` - inject and `Instrumentation` resource from the namespace. | ||
* `"my-instrumentation"` - name of `Instrumentation` CR instance. | ||
|
||
#### NodeJS | ||
|
||
The operator can inject OpenTelemetry [NodeJS auto-instrumentation](https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/metapackages/auto-instrumentations-node). | ||
The injection of the NodeJS instrumentation can be enabled by adding an annotation to the namespace, so that all pods within that namespace will get the instrumentation, or by adding the annotation to individual PodSpec objects, available as part of Deployment, Statefulset, and other resources. | ||
|
||
```bash | ||
instrumentation.opentelemetry.io/inject-nodejs: "true" | ||
``` | ||
|
||
The value can be | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we should DRY this, the semantics for the annotation are the same for all languages, no need to repeat this. |
||
* `"false"` - do not inject | ||
* `"true"` - inject and `Instrumentation` resource from the namespace. | ||
* `"my-instrumentation"` - name of `Instrumentation` CR instance. | ||
|
||
## Compatibility matrix | ||
|
||
|
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -23,15 +23,16 @@ import ( | |
const ( | ||
// annotationInjectJava indicates whether java auto-instrumentation should be injected or not. | ||
// Possible values are "true", "false" or "<Instrumentation>" name. | ||
annotationInjectJava = "instrumentation.opentelemetry.io/inject-java" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can you explain the intentions for changing this annotation? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah I'm happy to split out the annotation change if it makes sense. I found our current, SDK-agnostic (?) sdk.go uses this Java annotation. I need it to be able to support multiple languages and came up with this change. Any idea that could work better? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you please explain why we need two annotations? We can have There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think we want to allow multiple languages on the same pod as that shouldn't ever be a use case I think. While we could check the annotations and fail if there are multiple this split seemed to model more directly that only one language can be injected. I made language required without thinking too much but could definitely remove that to allow the control plane only injection if this otherwise makes sense. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't see much value of limiting it to only a single language, quite the opposite. What I don't like is to have two annotations from the UX perspective.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I believe these sort of annotations are always copy-pasted or frameworked, so I don't think there is a usability difference between the two patterns. If we think there is a use case for multiple annotations, then it makes sense but for example having both Great to hear everyone's ideas on the matter :) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Discussed with @pavolloffay offline and for now sticking to the current annotation format, but we'll need to address it soon. We can see the awkwardness here where common config can be injected from different instrumentations in a surprising way. Having separate annotations for the instrumentation config and what languages to use in the future will hopefully solve that. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. +1 let's discuss it in a separate issue. maybe there are other ways we can mitigate the issue |
||
annotationInjectJava = "instrumentation.opentelemetry.io/inject-java" | ||
annotationInjectNodeJS = "instrumentation.opentelemetry.io/inject-nodejs" | ||
) | ||
|
||
// annotationValue returns the effective annotationInjectJava value, based on the annotations from the pod and namespace. | ||
func annotationValue(ns metav1.ObjectMeta, pod metav1.ObjectMeta) string { | ||
func annotationValue(ns metav1.ObjectMeta, pod metav1.ObjectMeta, annotation string) string { | ||
// is the pod annotated with instructions to inject sidecars? is the namespace annotated? | ||
// if any of those is true, a sidecar might be desired. | ||
podAnnValue := pod.Annotations[annotationInjectJava] | ||
nsAnnValue := ns.Annotations[annotationInjectJava] | ||
podAnnValue := pod.Annotations[annotation] | ||
nsAnnValue := ns.Annotations[annotation] | ||
|
||
// if the namespace value is empty, the pod annotation should be used, whatever it is | ||
if len(nsAnnValue) == 0 { | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,68 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
package instrumentation | ||
|
||
import ( | ||
"github.com/go-logr/logr" | ||
corev1 "k8s.io/api/core/v1" | ||
|
||
"github.com/open-telemetry/opentelemetry-operator/apis/instrumentation/v1alpha1" | ||
) | ||
|
||
const ( | ||
envNodeOptions = "NODE_OPTIONS" | ||
nodeRequireArgument = " --require /otel-auto-instrumentation/autoinstrumentation.js" | ||
) | ||
|
||
func injectNodeJSSDK(logger logr.Logger, nodeJSSpec v1alpha1.NodeJSSpec, pod corev1.Pod) corev1.Pod { | ||
// caller checks if there is at least one container | ||
container := &pod.Spec.Containers[0] | ||
idx := getIndexOfEnv(container.Env, envNodeOptions) | ||
if idx == -1 { | ||
container.Env = append(container.Env, corev1.EnvVar{ | ||
Name: envNodeOptions, | ||
Value: nodeRequireArgument, | ||
}) | ||
} else if idx > -1 { | ||
if container.Env[idx].ValueFrom != nil { | ||
// TODO add to status object or submit it as an event | ||
logger.Info("Skipping NodeJS SDK injection, the container defines NODE_OPTIONS env var value via ValueFrom", "container", container.Name) | ||
return pod | ||
} | ||
container.Env[idx].Value = container.Env[idx].Value + nodeRequireArgument | ||
} | ||
container.VolumeMounts = append(container.VolumeMounts, corev1.VolumeMount{ | ||
Name: volumeName, | ||
MountPath: "/otel-auto-instrumentation", | ||
}) | ||
|
||
pod.Spec.Volumes = append(pod.Spec.Volumes, corev1.Volume{ | ||
Name: volumeName, | ||
VolumeSource: corev1.VolumeSource{ | ||
EmptyDir: &corev1.EmptyDirVolumeSource{}, | ||
}}) | ||
|
||
pod.Spec.InitContainers = append(pod.Spec.InitContainers, corev1.Container{ | ||
Name: initContainerName, | ||
Image: nodeJSSpec.Image, | ||
Command: []string{"cp", "-a", "/autoinstrumentation/.", "/otel-auto-instrumentation/"}, | ||
VolumeMounts: []corev1.VolumeMount{{ | ||
Name: volumeName, | ||
MountPath: "/otel-auto-instrumentation", | ||
}}, | ||
}) | ||
|
||
return pod | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there are empty spaces which should not be there