-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[pkg/stanza] Fix serverity range unmarshaling #16339
Merged
djaglowski
merged 1 commit into
open-telemetry:main
from
djaglowski:pkg-stanza-sev-range-fix
Nov 17, 2022
Merged
[pkg/stanza] Fix serverity range unmarshaling #16339
djaglowski
merged 1 commit into
open-telemetry:main
from
djaglowski:pkg-stanza-sev-range-fix
Nov 17, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
djaglowski
force-pushed
the
pkg-stanza-sev-range-fix
branch
2 times, most recently
from
November 17, 2022 14:28
dffbc96
to
7e679a3
Compare
This functionality was broken during refactoring of the unmarshaling logic. Previously, various yaml types could be used to represent a severity level. This was designed to be as open as possible to future implementations. However, it is now clear that only some keys are valid anyways. Namely, string and ints that represent OTel's native severity levels. Both strings and ints are now treated as strings when unmarshaling, which is fully compatible with the sev map lookup code.
djaglowski
force-pushed
the
pkg-stanza-sev-range-fix
branch
from
November 17, 2022 15:55
7e679a3
to
8648bd2
Compare
schmikei
approved these changes
Nov 17, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can confirm this works for me!
stdout
➜ ./bin/otelcontribcol_darwin_amd64 --config common_event_format_logs.yaml
2022/11/17 11:38:59 proto: duplicate proto type registered: jaeger.api_v2.PostSpansRequest
2022/11/17 11:38:59 proto: duplicate proto type registered: jaeger.api_v2.PostSpansResponse
2022-11-17T11:38:59.795-0500 info service/telemetry.go:110 Setting up own telemetry...
2022-11-17T11:38:59.795-0500 info service/telemetry.go:140 Serving Prometheus metrics {"address": ":8888", "level": "Basic"}
2022-11-17T11:38:59.795-0500 info components/components.go:30 In development component. May change in the future. {"kind": "exporter", "data_type": "logs", "name": "logging", "stability": "in development"}
2022-11-17T11:38:59.796-0500 info service/service.go:89 Starting otelcontribcol... {"Version": "v0.64.0-66-g8648bd23e2", "NumCPU": 12}
2022-11-17T11:38:59.796-0500 info extensions/extensions.go:41 Starting extensions...
2022-11-17T11:38:59.796-0500 info pipelines/pipelines.go:74 Starting exporters...
2022-11-17T11:38:59.796-0500 info pipelines/pipelines.go:78 Exporter is starting... {"kind": "exporter", "data_type": "logs", "name": "logging"}
2022-11-17T11:38:59.796-0500 info pipelines/pipelines.go:82 Exporter started. {"kind": "exporter", "data_type": "logs", "name": "logging"}
2022-11-17T11:38:59.796-0500 info pipelines/pipelines.go:86 Starting processors...
2022-11-17T11:38:59.796-0500 info pipelines/pipelines.go:98 Starting receivers...
2022-11-17T11:38:59.796-0500 info pipelines/pipelines.go:102 Receiver is starting... {"kind": "receiver", "name": "filelog", "pipeline": "logs"}
2022-11-17T11:38:59.796-0500 info adapter/receiver.go:55 Starting stanza receiver {"kind": "receiver", "name": "filelog", "pipeline": "logs"}
2022-11-17T11:38:59.796-0500 info pipelines/pipelines.go:106 Receiver started. {"kind": "receiver", "name": "filelog", "pipeline": "logs"}
2022-11-17T11:38:59.796-0500 info service/service.go:106 Everything is ready. Begin running and processing data.
2022-11-17T11:38:59.997-0500 info fileconsumer/file.go:159 Started watching file {"kind": "receiver", "name": "filelog", "pipeline": "logs", "component": "fileconsumer", "path": "/Users/keithschmitt/cel.log"}
2022-11-17T11:39:00.096-0500 info LogsExporter {"kind": "exporter", "data_type": "logs", "name": "logging", "#logs": 4}
config
receivers:
filelog:
include: [/Users/keithschmitt/cel.log]
start_at: beginning
operators:
- type: regex_parser
regex: '^(?P<timestamp>\w{3}\s+\d{1,2}\s+\d{2}:\d{2}:\d{2})\s+((?P<hostname>[^\s]+)\s+)?(?P<cef_headers>[\d\D]+)'
timestamp:
parse_from: attributes.timestamp
layout_type: gotime
layout: 'Jan 02 15:04:05'
location: UTC
- type: csv_parser
parse_from: attributes.cef_headers
header: 'version|device_vendor|device_product|device_version|signature_id|name|severity|extensions'
delimiter: "|"
- type: regex_parser
if: 'attributes.version != nil and attributes.version matches "CEF:[^\\|]*"'
regex: 'CEF:(?P<version>[^\|]*)'
parse_from: attributes.version
- type: severity_parser
if: 'attributes.severity != nil'
parse_from: attributes.severity
preset: none
mapping:
info:
- min: 0
max: 3
- low
warn:
- min: 4
max: 6
- medium
error:
- min: 7
max: 8
- high
fatal:
- min: 9
max: 10
- very-high
- id: add_log_type
type: add
field: attributes.log_type
value: cel
exporters:
logging:
service:
pipelines:
logs:
receivers: [filelog]
exporters: [logging]
Thanks @djaglowski 🚀
TylerHelmuth
approved these changes
Nov 17, 2022
JaredTan95
pushed a commit
to openinsight-proj/opentelemetry-collector-contrib
that referenced
this pull request
Nov 21, 2022
This functionality was broken during refactoring of the unmarshaling logic. Previously, various yaml types could be used to represent a severity level. This was designed to be as open as possible to future implementations. However, it is now clear that only some keys are valid anyways. Namely, string and ints that represent OTel's native severity levels. Both strings and ints are now treated as strings when unmarshaling, which is fully compatible with the sev map lookup code.
shalper2
pushed a commit
to shalper2/opentelemetry-collector-contrib
that referenced
this pull request
Dec 6, 2022
This functionality was broken during refactoring of the unmarshaling logic. Previously, various yaml types could be used to represent a severity level. This was designed to be as open as possible to future implementations. However, it is now clear that only some keys are valid anyways. Namely, string and ints that represent OTel's native severity levels. Both strings and ints are now treated as strings when unmarshaling, which is fully compatible with the sev map lookup code.
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This functionality was broken during refactoring of the unmarshaling logic.
Previously, various yaml types could be used to represent a severity level. This was designed to be as open as possible to future implementations. However, it is now clear that only some keys are valid anyways. Namely, string and ints that represent OTel's native severity levels. Both strings and ints are now treated as strings when unmarshaling, which is fully compatible with the sev map lookup code.
Resolves #16336