-
Notifications
You must be signed in to change notification settings - Fork 579
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add LogProcessor to baggagecopy #6277
Merged
Merged
Changes from 10 commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
3318c22
feat: add LogProcessor to baggagecopy
Jesse0Michael e90f7fc
Merge branch 'main' into baggagecopy
Jesse0Michael cb43c9b
Update CHANGELOG.md
Jesse0Michael 9a79e87
Merge branch 'baggagecopy' of ssh://github.com/Jesse0Michael/opentele…
Jesse0Michael 50c108d
Update processors/baggagecopy/doc.go
Jesse0Michael b71bae1
Update processors/baggagecopy/log_processor_test.go
Jesse0Michael 60688fb
test: remove comments in tests
Jesse0Michael f217e33
tests: refactor tests
Jesse0Michael ea6927f
Merge branch 'main' into baggagecopy
Jesse0Michael fe41e51
Merge branch 'main' into baggagecopy
Jesse0Michael a904d70
Merge branch 'main' into baggagecopy
Jesse0Michael e0a2c92
Update processors/baggagecopy/log_processor_test.go
Jesse0Michael 13a2b74
Merge branch 'main' into baggagecopy
Jesse0Michael File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,27 +1,37 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
// Package baggagecopy is an OpenTelemetry [Span Processor] that reads key/values | ||
// stored in [Baggage] in the starting span's parent context and adds them as | ||
// attributes to the span. | ||
// Package baggagecopy is an OpenTelemetry [Span Processor] and [Log Record Processor] | ||
// that reads key/values stored in [Baggage] in context provided to copy onto the span or log. | ||
// | ||
// The SpanProcessor retrieves [Baggage] from the starting span's parent context | ||
// and adds them as attributes to the span. | ||
// | ||
// Keys and values added to Baggage will appear on all subsequent child spans for | ||
// a trace within this service and will be propagated to external services via | ||
// propagation headers. | ||
// If the external services also have a Baggage span processor, the keys and | ||
// values will appear in those child spans as well. | ||
// | ||
// The LogProcessor retrieves [Baggage] from the the context provided when | ||
// emitting the log and adds them as attributes to the log. | ||
// Baggage may be propagated to external services via propagation headers. | ||
// and be used to add context to logs if the service also has a Baggage log processor. | ||
// | ||
// Do not put sensitive information in Baggage. | ||
// | ||
// # Usage | ||
// | ||
// Add the span processor when configuring the tracer provider. | ||
// | ||
// Add the log processor when configuring the logger provider. | ||
// | ||
// The convenience function [AllowAllBaggageKeys] is provided to | ||
// allow all baggage keys to be copied to the span. Alternatively, you can | ||
// allow all baggage keys to be copied. Alternatively, you can | ||
// provide a custom baggage key predicate to select which baggage keys you want | ||
// to copy. | ||
// | ||
// [Span Processor]: https://opentelemetry.io/docs/specs/otel/trace/sdk/#span-processor | ||
// [Log Record Processor]: https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecordprocessor | ||
// [Baggage]: https://opentelemetry.io/docs/specs/otel/api/baggage | ||
package baggagecopy // import "go.opentelemetry.io/contrib/processors/baggagecopy" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
package baggagecopy // import "go.opentelemetry.io/contrib/processors/baggagecopy" | ||
|
||
import ( | ||
"context" | ||
|
||
"go.opentelemetry.io/otel/baggage" | ||
api "go.opentelemetry.io/otel/log" | ||
"go.opentelemetry.io/otel/sdk/log" | ||
) | ||
|
||
// LogProcessor is a [log.Processor] implementation that adds baggage | ||
// members onto a log as attributes. | ||
type LogProcessor struct { | ||
filter Filter | ||
} | ||
|
||
var _ log.Processor = (*LogProcessor)(nil) | ||
|
||
// NewLogProcessor returns a new [LogProcessor]. | ||
// | ||
// The Baggage log processor adds attributes to a log record that are found | ||
// in Baggage in the parent context at the moment the log is emitted. | ||
// The passed filter determines which baggage members are added to the span. | ||
// | ||
// If filter is nil, all baggage members will be added. | ||
func NewLogProcessor(filter Filter) *LogProcessor { | ||
return &LogProcessor{ | ||
filter: filter, | ||
} | ||
} | ||
|
||
// OnEmit adds Baggage member to a log record as attributes that are pulled from | ||
// the Baggage found in ctx. Baggage members are filtered by the filter passed | ||
// to NewLogProcessor. | ||
func (processor LogProcessor) OnEmit(ctx context.Context, record *log.Record) error { | ||
filter := processor.filter | ||
if filter == nil { | ||
filter = AllowAllMembers | ||
} | ||
|
||
for _, member := range baggage.FromContext(ctx).Members() { | ||
if filter(member) { | ||
record.AddAttributes(api.String(member.Key(), member.Value())) | ||
} | ||
} | ||
|
||
return nil | ||
} | ||
|
||
// Shutdown is called when the [log.Processor] is shutting down and is a no-op for this processor. | ||
func (processor LogProcessor) Shutdown(context.Context) error { return nil } | ||
|
||
// ForceFlush is called to ensure all logs are flushed to the output and is a no-op for this processor. | ||
func (processor LogProcessor) ForceFlush(context.Context) error { return nil } |
Jesse0Michael marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,134 @@ | ||
// Copyright The OpenTelemetry Authors | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
package baggagecopy | ||
|
||
import ( | ||
"context" | ||
"regexp" | ||
"strings" | ||
"testing" | ||
|
||
"github.com/stretchr/testify/assert" | ||
"github.com/stretchr/testify/require" | ||
|
||
"go.opentelemetry.io/otel/baggage" | ||
api "go.opentelemetry.io/otel/log" | ||
"go.opentelemetry.io/otel/sdk/log" | ||
) | ||
|
||
var _ log.Processor = &processor{} | ||
|
||
type processor struct { | ||
records []*log.Record | ||
} | ||
|
||
func (p *processor) OnEmit(ctx context.Context, r *log.Record) error { | ||
p.records = append(p.records, r) | ||
return nil | ||
} | ||
|
||
func (p *processor) Shutdown(ctx context.Context) error { return nil } | ||
|
||
func (p *processor) ForceFlush(ctx context.Context) error { return nil } | ||
|
||
func NewTestProcessor() *processor { | ||
return &processor{} | ||
} | ||
|
||
func TestLogProcessor_OnEmit(t *testing.T) { | ||
Jesse0Michael marked this conversation as resolved.
Show resolved
Hide resolved
|
||
tests := []struct { | ||
name string | ||
baggage baggage.Baggage | ||
filter Filter | ||
want []api.KeyValue | ||
}{ | ||
{ | ||
name: "all baggage attributes", | ||
baggage: func() baggage.Baggage { | ||
b, _ := baggage.New() | ||
b = addEntryToBaggage(t, b, "baggage.test", "baggage value") | ||
return b | ||
}(), | ||
filter: AllowAllMembers, | ||
want: []api.KeyValue{api.String("baggage.test", "baggage value")}, | ||
}, | ||
{ | ||
name: "baggage attributes with prefix", | ||
baggage: func() baggage.Baggage { | ||
b, _ := baggage.New() | ||
b = addEntryToBaggage(t, b, "baggage.test", "baggage value") | ||
return b | ||
}(), | ||
filter: func(m baggage.Member) bool { | ||
return strings.HasPrefix(m.Key(), "baggage.") | ||
}, | ||
want: []api.KeyValue{api.String("baggage.test", "baggage value")}, | ||
}, | ||
{ | ||
name: "baggage attributes with regex", | ||
baggage: func() baggage.Baggage { | ||
b, _ := baggage.New() | ||
b = addEntryToBaggage(t, b, "baggage.test", "baggage value") | ||
return b | ||
}(), | ||
filter: func(m baggage.Member) bool { | ||
return regexp.MustCompile(`^baggage\..*`).MatchString(m.Key()) | ||
}, | ||
want: []api.KeyValue{api.String("baggage.test", "baggage value")}, | ||
}, | ||
{ | ||
name: "only adds baggage entries that match predicate", | ||
baggage: func() baggage.Baggage { | ||
b, _ := baggage.New() | ||
b = addEntryToBaggage(t, b, "baggage.test", "baggage value") | ||
b = addEntryToBaggage(t, b, "foo", "bar") | ||
return b | ||
}(), | ||
filter: func(m baggage.Member) bool { | ||
return m.Key() == "baggage.test" | ||
}, | ||
want: []api.KeyValue{api.String("baggage.test", "baggage value")}, | ||
}, | ||
} | ||
for _, tt := range tests { | ||
t.Run(tt.name, func(t *testing.T) { | ||
ctx := baggage.ContextWithBaggage(context.Background(), tt.baggage) | ||
|
||
wrapped := &processor{} | ||
lp := log.NewLoggerProvider( | ||
log.WithProcessor(NewLogProcessor(tt.filter)), | ||
log.WithProcessor(wrapped), | ||
) | ||
|
||
lp.Logger("test").Emit(ctx, api.Record{}) | ||
|
||
require.Len(t, wrapped.records, 1) | ||
require.Equal(t, len(tt.want), wrapped.records[0].AttributesLen()) | ||
|
||
var got []api.KeyValue | ||
wrapped.records[0].WalkAttributes(func(kv api.KeyValue) bool { | ||
got = append(got, kv) | ||
return true | ||
}) | ||
|
||
require.Equal(t, tt.want, got) | ||
}) | ||
} | ||
} | ||
|
||
func TestZeroLogProcessorNoPanic(t *testing.T) { | ||
lp := new(LogProcessor) | ||
|
||
m, err := baggage.NewMember("key", "val") | ||
require.NoError(t, err) | ||
b, err := baggage.New(m) | ||
require.NoError(t, err) | ||
|
||
ctx := baggage.ContextWithBaggage(context.Background(), b) | ||
assert.NotPanics(t, func() { | ||
_ = lp.OnEmit(ctx, &log.Record{}) | ||
_ = lp.Shutdown(ctx) | ||
_ = lp.ForceFlush(ctx) | ||
}) | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A filter can on be configured when creating the log processor so I think it would be better to set the default of AllowAllMembers in
NewLogProcessor
instead of on each call toOnEmit
.Then
OnEmit
doesn't need to worry about checking if filter is nil.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can certainly do that, but that does change the expectations slightly.
It adds a possible panic in the Zero value of the LogProcessor
something that we're testing against:
https://github.com/open-telemetry/opentelemetry-go-contrib/pull/6277/files#diff-b4c7636c6f14ae8c68b2f52c7de627b902307b42a5114af060c5f6e16b47c585R120-R134
That test is pulled from the code this mimics from the SpanProcessor.
I can move the nil check to
NewLogProcessor
, but then should I just ignore testing against panicing on the OnEmit method?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we want to guard against zero value use. Having the filter check here, similar to the span processor, seems appropriate.