Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix labels for Service Monitors #2878

Merged
merged 24 commits into from
May 20, 2024
Merged

Fix labels for Service Monitors #2878

Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
1625097
Create a separate Service Monitor when the Prometheus exporter is pre…
iblancasa Apr 22, 2024
78f1315
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
iblancasa Apr 25, 2024
9ccd87f
Improve changelog
iblancasa Apr 25, 2024
5f55a01
Fix prometheus-cr E2E test
iblancasa Apr 25, 2024
e3174b6
Remove unused target
iblancasa Apr 25, 2024
edfb463
Merge branch 'main' into bug/2877
iblancasa Apr 26, 2024
82fefa2
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
iblancasa Apr 29, 2024
28baa37
Add docstring
iblancasa Apr 29, 2024
7dcaadb
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
iblancasa May 3, 2024
2e143ee
Fix typo
iblancasa May 3, 2024
cc96402
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
iblancasa May 8, 2024
978dfa0
Merge branch 'main' into bug/2877
iblancasa May 8, 2024
f5b29d8
Change the label name
iblancasa May 9, 2024
fff9870
Merge branch 'bug/2877' of github.com:iblancasa/opentelemetry-operato…
iblancasa May 9, 2024
6432371
Merge branch 'main' into bug/2877
iblancasa May 13, 2024
2469162
Merge branch 'bug/2877' of github.com:iblancasa/opentelemetry-operato…
iblancasa May 13, 2024
0b2a439
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
iblancasa May 13, 2024
3712c43
Change changelog description
iblancasa May 13, 2024
743247b
Merge branch 'main' into bug/2877
iblancasa May 14, 2024
552cb3e
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
iblancasa May 17, 2024
720c6b1
Recover removed labels
iblancasa May 17, 2024
d1889d0
Merge branch 'main' of github.com:open-telemetry/opentelemetry-operat…
iblancasa May 17, 2024
cc84a10
Add missing labels
iblancasa May 17, 2024
e5b251c
Remove wrong labels
iblancasa May 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .chloggen/bug_2877.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: bug_fix

# The name of the component, or a single word describing the area of concern, (e.g. collector, target allocator, auto-instrumentation, opamp, github action)
component: collector

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: |
Create a Service Monitor for the collector Service when Prometheus exporter is used. A different Service Monitor is created for the monitoring service.
swiatekm marked this conversation as resolved.
Show resolved Hide resolved
This helps excluding the headless service (duplicating the metrics collection) and splits responsibilities between the two Service Monitors.
Now, the operator.opentelemetry.io/collector-service-type label is used to differentiate the services.

# One or more tracking issues related to the change
issues: [2877]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
151 changes: 77 additions & 74 deletions controllers/builder_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,12 +259,13 @@ service:
Name: "test-collector",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "base",
},
Annotations: nil,
},
Expand All @@ -284,13 +285,13 @@ service:
Name: "test-collector-headless",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-headless-service": "Exists",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "headless",
},
Annotations: map[string]string{
"service.beta.openshift.io/serving-cert-secret-name": "test-collector-headless-tls",
Expand All @@ -313,13 +314,13 @@ service:
Name: "test-collector-monitoring",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector-monitoring",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-monitoring-service": "Exists",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector-monitoring",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "monitoring",
},
Annotations: nil,
},
Expand Down Expand Up @@ -506,12 +507,13 @@ service:
Name: "test-collector",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "base",
},
Annotations: nil,
},
Expand All @@ -531,13 +533,13 @@ service:
Name: "test-collector-headless",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-headless-service": "Exists",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "headless",
},
Annotations: map[string]string{
"service.beta.openshift.io/serving-cert-secret-name": "test-collector-headless-tls",
Expand All @@ -560,13 +562,13 @@ service:
Name: "test-collector-monitoring",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector-monitoring",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-monitoring-service": "Exists",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector-monitoring",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "monitoring",
},
Annotations: nil,
},
Expand Down Expand Up @@ -774,12 +776,13 @@ service:
Name: "test-collector",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "base",
},
Annotations: nil,
},
Expand All @@ -799,13 +802,13 @@ service:
Name: "test-collector-headless",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-headless-service": "Exists",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "headless",
},
Annotations: map[string]string{
"service.beta.openshift.io/serving-cert-secret-name": "test-collector-headless-tls",
Expand All @@ -828,13 +831,13 @@ service:
Name: "test-collector-monitoring",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector-monitoring",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-monitoring-service": "Exists",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector-monitoring",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "monitoring",
},
Annotations: nil,
},
Expand Down Expand Up @@ -1311,13 +1314,13 @@ service:
Name: "test-collector-monitoring",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector-monitoring",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-monitoring-service": "Exists",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector-monitoring",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "monitoring",
},
Annotations: nil,
},
Expand Down Expand Up @@ -1705,13 +1708,13 @@ prometheus_cr:
Name: "test-collector-monitoring",
Namespace: "test",
Labels: map[string]string{
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector-monitoring",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-monitoring-service": "Exists",
"app.kubernetes.io/component": "opentelemetry-collector",
"app.kubernetes.io/instance": "test.test",
"app.kubernetes.io/managed-by": "opentelemetry-operator",
"app.kubernetes.io/name": "test-collector-monitoring",
"app.kubernetes.io/part-of": "opentelemetry",
"app.kubernetes.io/version": "latest",
"operator.opentelemetry.io/collector-service-type": "monitoring",
},
Annotations: nil,
},
Expand Down
2 changes: 1 addition & 1 deletion internal/manifests/collector/collector.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ func Build(params manifests.Params) ([]client.Object, error) {
if params.OtelCol.Spec.Mode == v1beta1.ModeSidecar {
manifestFactories = append(manifestFactories, manifests.Factory(PodMonitor))
} else {
manifestFactories = append(manifestFactories, manifests.Factory(ServiceMonitor))
manifestFactories = append(manifestFactories, manifests.Factory(ServiceMonitor), manifests.Factory(ServiceMonitorMonitoring))
}
}

Expand Down
37 changes: 21 additions & 16 deletions internal/manifests/collector/podmonitor.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,28 +31,14 @@ import (

// PodMonitor returns the pod monitor for the given instance.
func PodMonitor(params manifests.Params) (*monitoringv1.PodMonitor, error) {
if !params.OtelCol.Spec.Observability.Metrics.EnableMetrics {
params.Log.V(2).Info("Metrics disabled for this OTEL Collector",
"params.OtelCol.name", params.OtelCol.Name,
"params.OtelCol.namespace", params.OtelCol.Namespace,
)
return nil, nil
} else if params.Config.PrometheusCRAvailability() == prometheus.NotAvailable {
params.Log.V(1).Info("Cannot enable PodMonitor when prometheus CRDs are unavailable",
"params.OtelCol.name", params.OtelCol.Name,
"params.OtelCol.namespace", params.OtelCol.Namespace,
)
if !shouldCreatePodMonitor(params) {
return nil, nil
}
var pm monitoringv1.PodMonitor

if params.OtelCol.Spec.Mode != v1beta1.ModeSidecar {
return nil, nil
}
name := naming.PodMonitor(params.OtelCol.Name)
labels := manifestutils.Labels(params.OtelCol.ObjectMeta, name, params.OtelCol.Spec.Image, ComponentOpenTelemetryCollector, nil)
selectorLabels := manifestutils.SelectorLabels(params.OtelCol.ObjectMeta, ComponentOpenTelemetryCollector)
pm = monitoringv1.PodMonitor{
pm := monitoringv1.PodMonitor{
ObjectMeta: metav1.ObjectMeta{
Namespace: params.OtelCol.Namespace,
Name: name,
Expand Down Expand Up @@ -107,3 +93,22 @@ func metricsEndpointsFromConfig(logger logr.Logger, otelcol v1beta1.OpenTelemetr
}
return metricsEndpoints
}

func shouldCreatePodMonitor(params manifests.Params) bool {
l := params.Log.WithValues(
"params.OtelCol.name", params.OtelCol.Name,
"params.OtelCol.namespace", params.OtelCol.Namespace,
)

if !params.OtelCol.Spec.Observability.Metrics.EnableMetrics {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

personal opinion but this statement can be written in a more readable way

if params.OtelCol.Spec.Observability.Metrics.EnableMetrics &&  params.Config.PrometheusCRAvailability() == prometheus.NotAvailable && params.OtelCol.Spec.Mode == v1beta1.ModeSidecar{
  return true
}

l..V(2).Info("PodMonitor will not be created", "Metrics.enabledMetrics", "params.OtelCol.Spec.Observability.Metrics.EnableMetrics ", ....)
return false

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and I would move the logger creation close to returning false

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why a double condition + no log message is easier to read than the current approach?

The current is easy to follow and understand why the method returns false.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A single log message with all parameters makes it clear to understand the overall state.

The function has 2 states (true and false), aligning the condition to it makes it easier to understand

l.V(2).Info("Metrics disabled for this OTEL Collector. PodMonitor will not ve created")
return false
} else if params.Config.PrometheusCRAvailability() == prometheus.NotAvailable {
l.V(2).Info("Cannot enable PodMonitor when prometheus CRDs are unavailable")
return false
} else if params.OtelCol.Spec.Mode != v1beta1.ModeSidecar {
l.V(2).Info("Not using sidecar mode. PodMonitor will not be created")
return false
}
return true
}
23 changes: 17 additions & 6 deletions internal/manifests/collector/service.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,21 +29,31 @@ import (
"github.com/open-telemetry/opentelemetry-operator/internal/naming"
)

// headless and monitoring labels are to differentiate the headless/monitoring services from the clusterIP service.
// headless and monitoring labels are to differentiate the base/headless/monitoring services from the clusterIP service.
const (
headlessLabel = "operator.opentelemetry.io/collector-headless-service"
iblancasa marked this conversation as resolved.
Show resolved Hide resolved
monitoringLabel = "operator.opentelemetry.io/collector-monitoring-service"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iblancasa i think we should keep these around for now, mark them as deprecated and remove them in two+ versions rather than deleting them outright which would constitute a breaking change.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

valueExists = "Exists"
serviceTypeLabel = "operator.opentelemetry.io/collector-service-type"
)

type ServiceType int

const (
BaseServiceType ServiceType = iota
HeadlessServiceType
MonitoringServiceType
)

func (s ServiceType) String() string {
return [...]string{"base", "headless", "monitoring"}[s]
}

func HeadlessService(params manifests.Params) (*corev1.Service, error) {
h, err := Service(params)
if h == nil || err != nil {
return h, err
}

h.Name = naming.HeadlessService(params.OtelCol.Name)
h.Labels[headlessLabel] = valueExists
h.Labels[serviceTypeLabel] = HeadlessServiceType.String()

// copy to avoid modifying params.OtelCol.Annotations
annotations := map[string]string{
Expand All @@ -62,7 +72,7 @@ func MonitoringService(params manifests.Params) (*corev1.Service, error) {

name := naming.MonitoringService(params.OtelCol.Name)
labels := manifestutils.Labels(params.OtelCol.ObjectMeta, name, params.OtelCol.Spec.Image, ComponentOpenTelemetryCollector, []string{})
labels[monitoringLabel] = valueExists
labels[serviceTypeLabel] = MonitoringServiceType.String()

metricsPort, err := params.OtelCol.Spec.Config.Service.MetricsPort()
if err != nil {
Expand Down Expand Up @@ -90,6 +100,7 @@ func MonitoringService(params manifests.Params) (*corev1.Service, error) {
func Service(params manifests.Params) (*corev1.Service, error) {
name := naming.Service(params.OtelCol.Name)
labels := manifestutils.Labels(params.OtelCol.ObjectMeta, name, params.OtelCol.Spec.Image, ComponentOpenTelemetryCollector, []string{})
labels[serviceTypeLabel] = BaseServiceType.String()

out, err := params.OtelCol.Spec.Config.Yaml()
if err != nil {
Expand Down
Loading
Loading