Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update usage of logging exporter #2130

Merged
merged 6 commits into from
Oct 10, 2023
Merged

update usage of logging exporter #2130

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3acbcb9
update usage of logging exporter
Sep 15, 2023
357e8b6
add changelog
Sep 28, 2023
e5b6735
update collector version
Sep 29, 2023
ef2315b
update expected data
Oct 6, 2023
9053a66
fix test expectations
Oct 6, 2023
1e83004
Merge branch 'main' into codeboten/logging-debug
codeboten Oct 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .chloggen/codeboten_logging-debug.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. operator, target allocator, github action)
component: operator

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: updating the operator to use the Collector's debug exporter in replacement of the deprecated logging exporter

# One or more tracking issues related to the change
issues: [2130]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:
26 changes: 13 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,21 +48,21 @@ spec:
timeout: 10s

exporters:
logging:
debug:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should reflect this in a changelog? @open-telemetry/operator-approvers

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah we should probably changelog this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added changelog

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR changes just docs and e2e tests. Not sure why the changelog is needed. Users defining the logging exporter in their manifests will need to migrate explicitly.

@codeboten is the collector printing a warning if the logging exporter is used? The operator could parse the config and print a warning in the validating webhook.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pavolloffay if the logging exporter is used, it will print out a warning like this:

2023-09-29T08:00:43.970-0700	info	exporter/exporter.go:275	Deprecated component. Will be removed in future releases.	{"kind": "exporter", "data_type": "traces", "name": "logging"}
2023-09-29T08:00:43.970-0700	info	exporter/exporter.go:275	Deprecated component. Will be removed in future releases.	{"kind": "exporter", "data_type": "metrics", "name": "logging"}
2023-09-29T08:00:43.972-0700	info	exporter/exporter.go:275	Deprecated component. Will be removed in future releases.	{"kind": "exporter", "data_type": "logs", "name": "logging"}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This deprecation warning appears for any components that are marked as deprecated

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this sounds like a good followup task, but something we can do later


service:
pipelines:
traces:
receivers: [otlp]
processors: []
exporters: [logging]
exporters: [debug]
EOF
```

**_WARNING:_** Until the OpenTelemetry Collector format is stable, changes may be required in the above example to remain
compatible with the latest version of the OpenTelemetry Collector image being referenced.

This will create an OpenTelemetry Collector instance named `simplest`, exposing a `jaeger-grpc` port to consume spans from your instrumented applications and exporting those spans via `logging`, which writes the spans to the console (`stdout`) of the OpenTelemetry Collector instance that receives the span.
This will create an OpenTelemetry Collector instance named `simplest`, exposing a `jaeger-grpc` port to consume spans from your instrumented applications and exporting those spans via `debug`, which writes the spans to the console (`stdout`) of the OpenTelemetry Collector instance that receives the span.

The `config` node holds the `YAML` that should be passed down as-is to the underlying OpenTelemetry Collector instances. Refer to the [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector) documentation for a reference of the possible entries.

Expand Down Expand Up @@ -111,14 +111,14 @@ spec:
processors:

exporters:
logging:
debug:

service:
pipelines:
traces:
receivers: [jaeger]
processors: []
exporters: [logging]
exporters: [debug]
EOF

kubectl apply -f - <<EOF
Expand Down Expand Up @@ -449,14 +449,14 @@ spec:
replacement: $$1

exporters:
logging:
debug:

service:
pipelines:
metrics:
receivers: [prometheus]
processors: []
exporters: [logging]
exporters: [debug]
```
The usage of `$$` in the replacement keys in the example above is based on the information provided in the Prometheus receiver [README](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/prometheusreceiver/README.md) documentation, which states:
`Note: Since the collector configuration supports env variable substitution $ characters in your prometheus configuration are interpreted as environment variables. If you want to use $ characters in your prometheus configuration, you must escape them using $$.`
Expand All @@ -481,14 +481,14 @@ Behind the scenes, the OpenTelemetry Operator will convert the Collector’s con
replacement: $$1

exporters:
logging:
debug:

service:
pipelines:
metrics:
receivers: [prometheus]
processors: []
exporters: [logging]
exporters: [debug]
```

Note how the Operator removes any existing service discovery configurations (e.g., `static_configs`, `file_sd_configs`, etc.) from the `scrape_configs` section and adds an `http_sd_configs` configuration pointing to a Target Allocator instance it provisioned.
Expand Down Expand Up @@ -533,14 +533,14 @@ Operator add the necessary target allocator configuration automatically. This fe
collector_id: $POD_NAME

exporters:
logging:
debug:

service:
pipelines:
metrics:
receivers: [prometheus]
processors: []
exporters: [logging]
exporters: [debug]
```

This also allows for a more straightforward collector configuration for target discovery using prometheus-operator CRDs. See below for a minimal example:
Expand All @@ -563,14 +563,14 @@ spec:
config:

exporters:
logging:
debug:

service:
pipelines:
metrics:
receivers: [prometheus]
processors: []
exporters: [logging]
exporters: [debug]
```

## Compatibility matrix
Expand Down
2 changes: 1 addition & 1 deletion bundle/manifests/opentelemetry-operator.clusterserviceversion.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ metadata:
"name": "otel"
},
"spec": {
"config": "receivers:\n otlp:\n protocols: \n grpc:\n http:\n\nexporters:\n logging:\n\nservice:\n pipelines:\n traces:\n receivers: [otlp]\n exporters: [logging]\n"
"config": "receivers:\n otlp:\n protocols: \n grpc:\n http:\n\nexporters:\n debug:\n\nservice:\n pipelines:\n traces:\n receivers: [otlp]\n exporters: [debug]\n"
}
}
]
Expand Down
24 changes: 12 additions & 12 deletions cmd/operator-opamp-bridge/agent/agent_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ func TestAgent_onMessage(t *testing.T) {
},
},
status: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace405"),
LastRemoteConfigHash: []byte("good/testnamespace401"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_APPLIED,
},
},
Expand All @@ -187,7 +187,7 @@ func TestAgent_onMessage(t *testing.T) {
want: want{
contents: nil,
status: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("bad/testnamespace408"),
LastRemoteConfigHash: []byte("bad/testnamespace404"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_FAILED,
ErrorMessage: "yaml: line 16: could not find expected ':'",
},
Expand Down Expand Up @@ -216,7 +216,7 @@ func TestAgent_onMessage(t *testing.T) {
},
},
status: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace405"),
LastRemoteConfigHash: []byte("good/testnamespace401"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_APPLIED,
},
},
Expand All @@ -235,7 +235,7 @@ func TestAgent_onMessage(t *testing.T) {
want: want{
contents: nil,
status: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace405"),
LastRemoteConfigHash: []byte("good/testnamespace401"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_FAILED,
ErrorMessage: "Items in config are not allowed: [processors.batch]",
},
Expand All @@ -255,7 +255,7 @@ func TestAgent_onMessage(t *testing.T) {
want: want{
contents: nil,
status: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace405"),
LastRemoteConfigHash: []byte("good/testnamespace401"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_FAILED,
ErrorMessage: "Items in config are not allowed: [processors]",
},
Expand Down Expand Up @@ -288,7 +288,7 @@ func TestAgent_onMessage(t *testing.T) {
},
},
status: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace405"),
LastRemoteConfigHash: []byte("good/testnamespace401"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_APPLIED,
},
nextContents: map[string][]string{
Expand All @@ -303,7 +303,7 @@ func TestAgent_onMessage(t *testing.T) {
},
},
nextStatus: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace439"),
LastRemoteConfigHash: []byte("good/testnamespace435"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_APPLIED,
},
},
Expand Down Expand Up @@ -335,7 +335,7 @@ func TestAgent_onMessage(t *testing.T) {
},
},
status: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace405"),
LastRemoteConfigHash: []byte("good/testnamespace401"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_APPLIED,
},
nextContents: map[string][]string{
Expand All @@ -350,7 +350,7 @@ func TestAgent_onMessage(t *testing.T) {
},
},
nextStatus: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace408"), // The new hash should be of the bad config
LastRemoteConfigHash: []byte("good/testnamespace404"), // The new hash should be of the bad config
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_FAILED,
ErrorMessage: "yaml: line 16: could not find expected ':'",
},
Expand Down Expand Up @@ -383,7 +383,7 @@ func TestAgent_onMessage(t *testing.T) {
},
},
status: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace405"),
LastRemoteConfigHash: []byte("good/testnamespace401"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_APPLIED,
},
nextContents: map[string][]string{
Expand All @@ -405,7 +405,7 @@ func TestAgent_onMessage(t *testing.T) {
},
},
nextStatus: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace405other/testnamespace439"),
LastRemoteConfigHash: []byte("good/testnamespace401other/testnamespace435"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_APPLIED,
},
},
Expand Down Expand Up @@ -434,7 +434,7 @@ func TestAgent_onMessage(t *testing.T) {
},
},
status: &protobufs.RemoteConfigStatus{
LastRemoteConfigHash: []byte("good/testnamespace405"),
LastRemoteConfigHash: []byte("good/testnamespace401"),
Status: protobufs.RemoteConfigStatuses_RemoteConfigStatuses_APPLIED,
},
nextContents: map[string][]string{},
Expand Down
2 changes: 1 addition & 1 deletion cmd/operator-opamp-bridge/agent/testdata/agentbasiccomponentsallowed.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ components_allowed:
- memory_limiter
- batch
exporters:
- logging
- debug
2 changes: 1 addition & 1 deletion cmd/operator-opamp-bridge/agent/testdata/agentbatchnotallowed.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ components_allowed:
processors:
- memory_limiter
exporters:
- logging
- debug
2 changes: 1 addition & 1 deletion cmd/operator-opamp-bridge/agent/testdata/agentnoprocessorsallowed.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ components_allowed:
receivers:
- otlp
exporters:
- logging
- debug
4 changes: 2 additions & 2 deletions cmd/operator-opamp-bridge/agent/testdata/basic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ config: |
timeout: 10s

exporters:
logging:
debug:

service:
pipelines:
traces:
receivers: [otlp]
processors: []
exporters: [logging]
exporters: [debug]
4 changes: 2 additions & 2 deletions cmd/operator-opamp-bridge/agent/testdata/invalid.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ config: |

GARBAGE
exporters:
logging:
debug:
service:
pipelines:
traces:
receivers: [otlp]
processors: []
exporters: [logging]
exporters: [debug]
4 changes: 2 additions & 2 deletions cmd/operator-opamp-bridge/agent/testdata/updated.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ config: |
timeout: 10s

exporters:
logging:
debug:

service:
pipelines:
traces:
receivers: [otlp]
processors: [memory_limiter, batch]
exporters: [logging]
exporters: [debug]
replicas: 3
4 changes: 2 additions & 2 deletions cmd/operator-opamp-bridge/operator/testdata/collector.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ config: |
timeout: 10s

exporters:
logging:
debug:

service:
pipelines:
traces:
receivers: [otlp]
processors: []
exporters: [logging]
exporters: [debug]
4 changes: 2 additions & 2 deletions cmd/operator-opamp-bridge/operator/testdata/invalid-collector.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ config: |
timeout: 10s
GARBAGE
exporters:
logging:
debug:

service:
pipelines:
traces:
receivers: [otlp]
processors: []
exporters: [logging]
exporters: [debug]
4 changes: 2 additions & 2 deletions cmd/operator-opamp-bridge/operator/testdata/updated-collector.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ config: |
timeout: 10s

exporters:
logging:
debug:

service:
pipelines:
traces:
receivers: [otlp]
processors: [memory_limiter, batch]
exporters: [logging]
exporters: [debug]
4 changes: 2 additions & 2 deletions cmd/otel-allocator/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,14 @@ spec:
- targets: [ '0.0.0.0:8888' ]

exporters:
logging:
debug:

service:
pipelines:
metrics:
receivers: [prometheus]
processors: []
exporters: [logging]
exporters: [debug]
```

In essence, Prometheus Receiver configs are overridden with a `http_sd_config` directive that points to the
Expand Down
4 changes: 2 additions & 2 deletions config/samples/core_v1alpha1_opentelemetrycollector.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ spec:
http:

exporters:
logging:
debug:

service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging]
exporters: [debug]
4 changes: 2 additions & 2 deletions config/samples/sidecar.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ spec:
http:

exporters:
logging:
debug:

service:
pipelines:
traces:
receivers: [otlp]
exporters: [logging]
exporters: [debug]
Loading
Loading