Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add autoinstrumentation of NodeJS #507

Merged
merged 20 commits into from
Nov 10, 2021
Merged

Add autoinstrumentation of NodeJS #507

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
ba0bede
Add autoinstrumentation of NodeJS
Nov 5, 2021
8a3770a
Drift
Nov 5, 2021
c54a5bc
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
Nov 9, 2021
ae74544
Switch to single annotation
Nov 9, 2021
3c5c95e
Fix merge
Nov 9, 2021
21d5433
Fix
Nov 9, 2021
9b7a81e
Hurts
Nov 9, 2021
b0239d6
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
Nov 9, 2021
7cc5182
Format
Nov 9, 2021
fb23588
Revert autogen
Nov 9, 2021
a302e31
Drift
Nov 9, 2021
1bbe523
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
Nov 10, 2021
293a8e0
Revert autogen
Nov 10, 2021
b02d1d9
autogen
Nov 10, 2021
0b72fa3
Drift
Nov 10, 2021
50785ae
Add doc
Nov 10, 2021
cff5521
e2e
Nov 10, 2021
d063917
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
Nov 10, 2021
d51b0aa
Less doc
Nov 10, 2021
b4b8cbb
Cleanup
Nov 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions api/instrumentation/v1alpha1/instrumentation_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ type InstrumentationSpec struct {
// +optional
// +operator-sdk:gen-csv:customresourcedefinitions.specDescriptors=true
Java JavaSpec `json:"java,omitempty"`

// NodeJS defines configuration for nodejs auto-instrumentation.
// +optional
// +operator-sdk:gen-csv:customresourcedefinitions.specDescriptors=true
NodeJS NodeJSSpec `json:"nodejs,omitempty"`
}

// JavaSpec defines Java SDK and instrumentation configuration.
Expand All @@ -37,6 +42,14 @@ type JavaSpec struct {
Image string `json:"image,omitempty"`
}

// NodeJSSpec defines NodeJS SDK and instrumentation configuration.
type NodeJSSpec struct {
// Image is a container image with NodeJS SDK and autoinstrumentation.
// +optional
// +operator-sdk:gen-csv:customresourcedefinitions.specDescriptors=true
Image string `json:"image,omitempty"`
}

// Exporter defines OTLP exporter configuration.
type Exporter struct {
// Endpoint is address of the collector with OTLP endpoint.
Expand Down
16 changes: 16 additions & 0 deletions api/instrumentation/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 7 additions & 0 deletions bundle/manifests/opentelemetry.io_instrumentations.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,13 @@ spec:
description: Image is a container image with javaagent JAR.
type: string
type: object
nodejs:
description: NodeJS defines configuration for nodejs auto-instrumentation.
properties:
image:
description: Image is a container image with NodeJS SDK and autoinstrumentation.
type: string
type: object
type: object
status:
description: InstrumentationStatus defines status of the instrumentation.
Expand Down
7 changes: 7 additions & 0 deletions config/crd/bases/opentelemetry.io_instrumentations.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,13 @@ spec:
description: Image is a container image with javaagent JAR.
type: string
type: object
nodejs:
description: NodeJS defines configuration for nodejs auto-instrumentation.
properties:
image:
description: Image is a container image with NodeJS SDK and autoinstrumentation.
type: string
type: object
type: object
status:
description: InstrumentationStatus defines status of the instrumentation.
Expand Down
9 changes: 5 additions & 4 deletions pkg/instrumentation/annotation.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,16 @@ import (
const (
// annotationInjectJava indicates whether java auto-instrumentation should be injected or not.
// Possible values are "true", "false" or "<Instrumentation>" name.
annotationInjectJava = "instrumentation.opentelemetry.io/inject-java"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you explain the intentions for changing this annotation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'm happy to split out the annotation change if it makes sense.

I found our current, SDK-agnostic (?) sdk.go uses this Java annotation. I need it to be able to support multiple languages and came up with this change. Any idea that could work better?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please explain why we need two annotations?

We can have "instrumentation.opentelemetry.io/inject-java", "instrumentation.opentelemetry.io/inject-node" and then `"instrumentation.opentelemetry.io/inject" - to just inject the basic configuration (users could use the operator as a control plane to manage instrumentations).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want to allow multiple languages on the same pod as that shouldn't ever be a use case I think. While we could check the annotations and fail if there are multiple this split seemed to model more directly that only one language can be injected.

I made language required without thinking too much but could definitely remove that to allow the control plane only injection if this otherwise makes sense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see much value of limiting it to only a single language, quite the opposite.

What I don't like is to have two annotations from the UX perspective.

	annotationInject   = "instrumentation.opentelemetry.io/inject"
	annotationLanguage = "instrumentation.opentelemetry.io/language"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe these sort of annotations are always copy-pasted or frameworked, so I don't think there is a usability difference between the two patterns. If we think there is a use case for multiple annotations, then it makes sense but for example having both otel.inject=true and otel.inject-java=true on the same pod would be redundant and I think that leads to some cognitive dissonance.

Great to hear everyone's ideas on the matter :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed with @pavolloffay offline and for now sticking to the current annotation format, but we'll need to address it soon. We can see the awkwardness here

https://github.com/open-telemetry/opentelemetry-operator/pull/507/files#diff-92f790ce871022e5e7de195a2f96235dac6ac3fb04337974f8584c7c020a67cbR48

where common config can be injected from different instrumentations in a surprising way. Having separate annotations for the instrumentation config and what languages to use in the future will hopefully solve that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 let's discuss it in a separate issue. maybe there are other ways we can mitigate the issue

annotationInject = "instrumentation.opentelemetry.io/inject"
annotationLanguage = "instrumentation.opentelemetry.io/language"
)

// annotationValue returns the effective annotationInjectJava value, based on the annotations from the pod and namespace.
func annotationValue(ns metav1.ObjectMeta, pod metav1.ObjectMeta) string {
func annotationValue(ns metav1.ObjectMeta, pod metav1.ObjectMeta, annotation string) string {
// is the pod annotated with instructions to inject sidecars? is the namespace annotated?
// if any of those is true, a sidecar might be desired.
podAnnValue := pod.Annotations[annotationInjectJava]
nsAnnValue := ns.Annotations[annotationInjectJava]
podAnnValue := pod.Annotations[annotation]
nsAnnValue := ns.Annotations[annotation]

// if the namespace value is empty, the pod annotation should be used, whatever it is
if len(nsAnnValue) == 0 {
Expand Down
22 changes: 11 additions & 11 deletions pkg/instrumentation/annotation_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,14 @@ func TestEffectiveAnnotationValue(t *testing.T) {
corev1.Pod{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
annotationInjectJava: "true",
annotationInject: "true",
},
},
},
corev1.Namespace{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
annotationInjectJava: "false",
annotationInject: "false",
},
},
},
Expand All @@ -54,14 +54,14 @@ func TestEffectiveAnnotationValue(t *testing.T) {
corev1.Pod{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
annotationInjectJava: "true",
annotationInject: "true",
},
},
},
corev1.Namespace{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
annotationInjectJava: "some-instance",
annotationInject: "some-instance",
},
},
},
Expand All @@ -73,14 +73,14 @@ func TestEffectiveAnnotationValue(t *testing.T) {
corev1.Pod{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
annotationInjectJava: "some-instance-from-pod",
annotationInject: "some-instance-from-pod",
},
},
},
corev1.Namespace{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
annotationInjectJava: "some-instance",
annotationInject: "some-instance",
},
},
},
Expand All @@ -92,14 +92,14 @@ func TestEffectiveAnnotationValue(t *testing.T) {
corev1.Pod{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
annotationInjectJava: "false",
annotationInject: "false",
},
},
},
corev1.Namespace{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
annotationInjectJava: "some-instance",
annotationInject: "some-instance",
},
},
},
Expand All @@ -112,7 +112,7 @@ func TestEffectiveAnnotationValue(t *testing.T) {
corev1.Namespace{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
annotationInjectJava: "some-instance",
annotationInject: "some-instance",
},
},
},
Expand All @@ -124,7 +124,7 @@ func TestEffectiveAnnotationValue(t *testing.T) {
corev1.Pod{
ObjectMeta: metav1.ObjectMeta{
Annotations: map[string]string{
annotationInjectJava: "true",
annotationInject: "true",
},
},
},
Expand All @@ -133,7 +133,7 @@ func TestEffectiveAnnotationValue(t *testing.T) {
} {
t.Run(tt.desc, func(t *testing.T) {
// test
annValue := annotationValue(tt.ns.ObjectMeta, tt.pod.ObjectMeta)
annValue := annotationValue(tt.ns.ObjectMeta, tt.pod.ObjectMeta, annotationInject)

// verify
assert.Equal(t, tt.expected, annValue)
Expand Down
68 changes: 68 additions & 0 deletions pkg/instrumentation/nodejs.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package instrumentation

import (
"github.com/go-logr/logr"
corev1 "k8s.io/api/core/v1"

"github.com/open-telemetry/opentelemetry-operator/api/instrumentation/v1alpha1"
)

const (
envNodeOptions = "NODE_OPTIONS"
nodeRequireArgument = " --require /otel-auto-instrumentation/autoinstrumentation.js"
)

func injectNodeJSSDK(logger logr.Logger, nodeJSSpec v1alpha1.NodeJSSpec, pod corev1.Pod) corev1.Pod {
// caller checks if there is at least one container
container := &pod.Spec.Containers[0]
idx := getIndexOfEnv(container.Env, envNodeOptions)
if idx == -1 {
container.Env = append(container.Env, corev1.EnvVar{
Name: envNodeOptions,
Value: nodeRequireArgument,
})
} else if idx > -1 {
if container.Env[idx].ValueFrom != nil {
// TODO add to status object or submit it as an event
logger.Info("Skipping NodeJS SDK injection, the container defines NODE_OPTIONS env var value via ValueFrom", "container", container.Name)
return pod
}
container.Env[idx].Value = container.Env[idx].Value + nodeRequireArgument
}
container.VolumeMounts = append(container.VolumeMounts, corev1.VolumeMount{
Name: volumeName,
MountPath: "/otel-auto-instrumentation",
})

pod.Spec.Volumes = append(pod.Spec.Volumes, corev1.Volume{
Name: volumeName,
VolumeSource: corev1.VolumeSource{
EmptyDir: &corev1.EmptyDirVolumeSource{},
}})

pod.Spec.InitContainers = append(pod.Spec.InitContainers, corev1.Container{
Name: initContainerName,
Image: nodeJSSpec.Image,
Command: []string{"cp", "-a", "/autoinstrumentation/.", "/otel-auto-instrumentation/"},
VolumeMounts: []corev1.VolumeMount{{
Name: volumeName,
MountPath: "/otel-auto-instrumentation",
}},
})

return pod
}
Loading