Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add honeycomb marker exporter structure #27001

Merged
merged 76 commits into from
Oct 12, 2023
Merged
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
7a00beb
feat: add honeycomb marker exporter structure
fchikwekwe Sep 19, 2023
3304fe5
Merge branch 'main' into feat/marker-exporter
fchikwekwe Sep 19, 2023
c2d6549
fix: remove start and stop
fchikwekwe Sep 21, 2023
34c66e7
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Sep 21, 2023
3a21318
feat: add honeycomb marker exporter
fchikwekwe Sep 25, 2023
8a1393f
feat: add generated files for component
fchikwekwe Sep 25, 2023
3decc4c
Merge branch 'main' into feat/marker-exporter
fchikwekwe Sep 25, 2023
a82ba30
doc: add changelog
fchikwekwe Sep 25, 2023
c892bca
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Sep 25, 2023
cb9efe8
Merge branch 'main' into feat/marker-exporter
fchikwekwe Sep 25, 2023
09a4438
fix: remove presets and allow configurable markers and rules
fchikwekwe Sep 29, 2023
d7e8882
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 2, 2023
cdf882a
fix: fix config examples and README to use configurable markers
fchikwekwe Oct 2, 2023
537b227
fix: parse logs through transform processor parser funcs
fchikwekwe Oct 3, 2023
52f85c8
Update exporter/honeycombmarkerexporter/factory.go
fchikwekwe Oct 3, 2023
792bdce
Update exporter/honeycombmarkerexporter/factory.go
fchikwekwe Oct 3, 2023
1fb07b0
Update exporter/honeycombmarkerexporter/metadata.yaml
fchikwekwe Oct 3, 2023
311a420
fix: use configopaque
fchikwekwe Oct 3, 2023
660d6a1
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 3, 2023
a49faf1
fix: add support for checking log and resource conditions
fchikwekwe Oct 5, 2023
41a0858
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 5, 2023
c553616
chore: update changelog
fchikwekwe Oct 5, 2023
4dfbff3
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 5, 2023
f92d3df
refactor: rename exporter dir
fchikwekwe Oct 5, 2023
abdad4a
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 5, 2023
44cbc2c
fix: revert changes to filter processor
fchikwekwe Oct 5, 2023
91c90ba
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 5, 2023
414367b
fix: revert changes to filter processor
fchikwekwe Oct 5, 2023
372171c
fix: remove comments
fchikwekwe Oct 5, 2023
a8c920a
Update exporter/honeycombexporter/README.md
fchikwekwe Oct 5, 2023
03ba0bf
Update exporter/honeycombexporter/metadata.yaml
fchikwekwe Oct 6, 2023
ec08d37
Update .chloggen/feat_marker-exporter.yaml
fchikwekwe Oct 6, 2023
f21943a
test: update config test
fchikwekwe Oct 6, 2023
73453ed
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 6, 2023
5c55fd6
test: add more config test cases
fchikwekwe Oct 6, 2023
71a9d77
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 6, 2023
fc8970b
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 6, 2023
bab5a8b
Update exporter/honeycombexporter/factory.go
fchikwekwe Oct 6, 2023
3981adc
test: fix no api url case
fchikwekwe Oct 10, 2023
f762563
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 10, 2023
782ce2c
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 10, 2023
fe60983
fix: update license in config test
fchikwekwe Oct 10, 2023
63800b5
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 10, 2023
1cf336f
fix: rename exporter pkg
fchikwekwe Oct 10, 2023
c0014cc
fix: comply with linter
fchikwekwe Oct 10, 2023
7e8da90
fix: update url fields in config test
fchikwekwe Oct 10, 2023
78b1b0a
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 10, 2023
b2306c2
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 10, 2023
9573c13
chore: add go mod file
fchikwekwe Oct 10, 2023
c75e2e8
chore: update deps
fchikwekwe Oct 10, 2023
e511eb6
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 10, 2023
c29a3a8
chore: update deps and comply with linter
fchikwekwe Oct 10, 2023
c16d9cd
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 10, 2023
5c856ce
Update exporter/honeycombmarkerexporter/go.mod
TylerHelmuth Oct 10, 2023
09a68e5
chore: fix go mod
fchikwekwe Oct 10, 2023
50efce0
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 10, 2023
35f2be8
chore: update deps
fchikwekwe Oct 10, 2023
19a1cf6
chore: update crosslink
fchikwekwe Oct 10, 2023
e8b2fce
chore: regenerate files
fchikwekwe Oct 10, 2023
86f43cb
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 10, 2023
7d6d0c5
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 11, 2023
278c548
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 11, 2023
9183300
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 11, 2023
f52e8c7
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 11, 2023
026926e
chore: update go mod
fchikwekwe Oct 11, 2023
4b21e62
chore: run go mod tidy and update configopaue version
fchikwekwe Oct 11, 2023
89e659c
chore: regen dependabot yml
fchikwekwe Oct 11, 2023
e38fce0
chore: regen issue template files
fchikwekwe Oct 11, 2023
927660f
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 11, 2023
d14c5df
chore: regen bug report
fchikwekwe Oct 11, 2023
237d03f
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 11, 2023
cdf7a83
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 11, 2023
d81ab3b
fix: change markertype and markercolor to type and color
fchikwekwe Oct 11, 2023
d1048eb
Merge branch 'feat/marker-exporter' of https://github.com/fchikwekwe/…
fchikwekwe Oct 11, 2023
dcc220e
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 11, 2023
61d9f7d
Merge branch 'main' into feat/marker-exporter
fchikwekwe Oct 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .chloggen/feat_marker-exporter.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: 'new_component'

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: 'honeycombmarkerexporter'

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: "This component will export markers to be consumed by the Honeycomb Markers API to highlight user events"

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [26653]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: []
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ exporter/fileexporter/ @open-te
exporter/googlecloudexporter/ @open-telemetry/collector-contrib-approvers @aabmass @dashpole @jsuereth @punya @damemi @psx95
exporter/googlecloudpubsubexporter/ @open-telemetry/collector-contrib-approvers @alexvanboxel
exporter/googlemanagedprometheusexporter/ @open-telemetry/collector-contrib-approvers @aabmass @dashpole @jsuereth @punya @damemi @psx95
exporter/honeycombmarkerexporter/ @open-telemetry/collector-contrib-approvers @fchikwekwe @TylerHelmuth
exporter/influxdbexporter/ @open-telemetry/collector-contrib-approvers @jacobmarble
exporter/instanaexporter/ @open-telemetry/collector-contrib-approvers @jpkrohling @hickeyma
exporter/kafkaexporter/ @open-telemetry/collector-contrib-approvers @pavolloffay @MovieStoreGuy
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ body:
- exporter/googlecloud
- exporter/googlecloudpubsub
- exporter/googlemanagedprometheus
- exporter/honeycombmarkerexporter
- exporter/influxdb
- exporter/instana
- exporter/kafka
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ body:
- exporter/googlecloud
- exporter/googlecloudpubsub
- exporter/googlemanagedprometheus
- exporter/honeycombmarkerexporter
- exporter/influxdb
- exporter/instana
- exporter/kafka
Expand Down
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/other.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ body:
- exporter/googlecloud
- exporter/googlecloudpubsub
- exporter/googlemanagedprometheus
- exporter/honeycombmarkerexporter
- exporter/influxdb
- exporter/instana
- exporter/kafka
Expand Down
1 change: 1 addition & 0 deletions exporter/honeycombmarkerexporter/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
46 changes: 46 additions & 0 deletions exporter/honeycombmarkerexporter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
@@ -0,0 +1,18 @@
# Honeycomb Marker Exporter

This exporter allows creating markers, via the Honeycomb Markers API, based on the look of incoming telemetry.

The following configuration options are supported:

* `api_key` (Required): This is the API key (also called Write Key) for your Honeycomb account.
* `api_url` (Required): You can set the hostname to send marker data to.
* `markers` (Required): This specifies the exact configuration to create an event marker.
* `markertype`: Specifies the marker type. Markers with the same type will appear in the same color in Honeycomb. MarkerType or MarkerColor should be set.
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
* `markercolor`: Specifies the marker color. Will only be used if MarkerType is not set.
* `messagefield`: This is the attribute that will be used as the message. If necessary the value will be converted to a string.
* `urlfield`: This is the attribute that will be used as the url. If necessary the value will be converted to a string.
* `rules`: This is a list of OTTL rules that determine when to create an event marker.
* `resourceconditions`: A list of ottlresource conditions that determine a match
* `logconditions`: A list of ottllog conditions that determine a match
Example:

```yaml
exporters:
honeycomb:
api_key: "my-api-key"
api_url: "https://api.testhost.io"
markers:
- markertype: "fooType",
messagefield: "test message",
urlfield: "https://api.testhost.io",
rules:
- resourceconditions:
- "ResourceCondition1"
- "ResourceCondition2"
- logconditions:
- "LogCondition1"
- "LogCondition2"
- markertype: "barType",
messagefield: "another test message",
urlfield: "https://api.testhost.io",
rules:
- resourceconditions:
- "ResourceCondition1"
- "ResourceCondition2"
- logconditions:
- "LogCondition1"
- "LogCondition2"
```
88 changes: 88 additions & 0 deletions exporter/honeycombmarkerexporter/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package honeycombmarkerexporter // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/honeycombmarkerexporter"

import (
"fmt"
"github.com/open-telemetry/opentelemetry-collector-contrib/internal/filter/filterottl"
"github.com/open-telemetry/opentelemetry-collector-contrib/pkg/ottl"
"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/configopaque"
"go.uber.org/zap"
)

// Config defines configuration for the Honeycomb Marker exporter.
type Config struct {
// APIKey is the authentication token associated with the Honeycomb account.
APIKey configopaque.String `mapstructure:"api_key"`

// API URL to use (defaults to https://api.honeycomb.io)
APIURL string `mapstructure:"api_url"`

// Markers is the list of markers to create
Markers []marker `mapstructure:"markers"`
}

type marker struct {
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
// MarkerType defines the type of marker. Markers with the same type appear in Honeycomb with the same color
MarkerType string `mapstructure:"type"`

// MarkerColor is the color of the marker. Will only be used if the MarkerType does not already exist.
MarkerColor string `mapstructure:"color"`

// MessageField is the attribute that will be used as the message.
// If necessary the value will be converted to a string.
MessageField string `mapstructure:"message_field"`

// URLField is the attribute that will be used as the url.
// If necessary the value will be converted to a string.
URLField string `mapstructure:"url_field"`

// Rules are the OTTL rules that determine when a piece of telemetry should be turned into a Marker
Rules Rules `mapstructure:"rules"`
}

type Rules struct {
// ResourceConditions is the list of ottlresource conditions that determine a match
ResourceConditions []string `mapstructure:"resource_conditions"`

// LogConditions is the list of ottllog conditions that determine a match
LogConditions []string `mapstructure:"log_conditions"`
}

var defaultCfg = CreateDefaultConfig().(*Config)

func (cfg *Config) Validate() error {
TylerHelmuth marked this conversation as resolved.
Show resolved Hide resolved
if cfg == nil {
cfg = defaultCfg
}

if cfg.APIKey == "" {
return fmt.Errorf("invalid API Key")
}

if len(cfg.Markers) != 0 {
for _, m := range cfg.Markers {
if len(m.Rules.ResourceConditions) == 0 && len(m.Rules.LogConditions) == 0 {
return fmt.Errorf("no rules supplied for marker %v", m)
}

_, err := filterottl.NewBoolExprForResource(m.Rules.ResourceConditions, filterottl.StandardResourceFuncs(), ottl.PropagateError, component.TelemetrySettings{Logger: zap.NewNop()})
if err != nil {
return err
}

_, err = filterottl.NewBoolExprForLog(m.Rules.LogConditions, filterottl.StandardLogFuncs(), ottl.PropagateError, component.TelemetrySettings{Logger: zap.NewNop()})
if err != nil {
return err
}
}
} else {
return fmt.Errorf("no markers supplied")
}

return nil
}

var _ component.Config = (*Config)(nil)
112 changes: 112 additions & 0 deletions exporter/honeycombmarkerexporter/config_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package honeycombmarkerexporter

import (
"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/honeycombmarkerexporter/internal/metadata"
"path/filepath"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/confmap/confmaptest"
)

func TestLoadConfig(t *testing.T) {
t.Parallel()

cm, err := confmaptest.LoadConf(filepath.Join("testdata", "config.yaml"))
require.NoError(t, err)

tests := []struct {
id component.ID
expected component.Config
}{
{
id: component.NewIDWithName("honeycomb", ""),
expected: &Config{
APIKey: "test-apikey",
APIURL: "https://api.testhost.io",
Markers: []marker{
{
MarkerType: "fooType",
MessageField: "test message",
URLField: "https://api.testhost.io",
Rules: Rules{
ResourceConditions: []string{
`IsMatch(attributes["test"], ".*")`,
},
LogConditions: []string{
`body == "test"`,
},
},
},
},
},
},
{
id: component.NewIDWithName("honeycomb", "color_no_type"),
expected: &Config{
APIKey: "test-apikey",
APIURL: "https://api.testhost.io",
Markers: []marker{
{
MarkerColor: "green",
MessageField: "test message",
URLField: "https://api.testhost.io",
Rules: Rules{
ResourceConditions: []string{
`IsMatch(attributes["test"], ".*")`,
},
LogConditions: []string{
`body == "test"`,
},
},
},
},
},
},
{
id: component.NewIDWithName(metadata.Type, "bad_syntax_log"),
},
{
id: component.NewIDWithName(metadata.Type, "no_conditions"),
},
{
id: component.NewIDWithName(metadata.Type, "no_api_key"),
},
{
id: component.NewIDWithName(metadata.Type, "no_markers_supplied"),
},
}

for _, tt := range tests {
t.Run(tt.id.String(), func(t *testing.T) {
factory := NewFactory()
cfg := factory.CreateDefaultConfig()

sub, err := cm.Sub(tt.id.String())
require.NoError(t, err)
require.NoError(t, component.UnmarshalConfig(sub, cfg))

if tt.expected == nil {
err = component.ValidateConfig(cfg)
assert.Error(t, err)
return
}

assert.NoError(t, component.ValidateConfig(cfg))
assert.Equal(t, tt.expected, cfg)
})
}
}

func withDefaultConfig(fns ...func(*Config)) *Config {
cfg := CreateDefaultConfig().(*Config)
for _, fn := range fns {
fn(cfg)
}
return cfg
}
7 changes: 7 additions & 0 deletions exporter/honeycombmarkerexporter/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

//go:generate mdatagen metadata.yaml

// Package honeycombmarkerexporter exports marker data to Honeycomb.
package honeycombmarkerexporter // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/honeycombmarkerexporter"
47 changes: 47 additions & 0 deletions exporter/honeycombmarkerexporter/factory.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package honeycombmarkerexporter // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/honeycombmarkerexporter"

import (
"context"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/exporter"
"go.opentelemetry.io/collector/exporter/exporterhelper"

"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/honeycombmarkerexporter/internal/metadata"
)

func NewFactory() exporter.Factory {
return exporter.NewFactory(
metadata.Type,
CreateDefaultConfig,
exporter.WithLogs(createLogsExporter, metadata.LogsStability),
)
}

func CreateDefaultConfig() component.Config {
return &Config{
APIKey: "",
APIURL: "api.honeycomb.io:443",
Markers: []marker{},
}
}

func createLogsExporter(
ctx context.Context,
set exporter.CreateSettings,
cfg component.Config,
) (exporter.Logs, error) {
cf := cfg.(*Config)

exporter := newLogsExporter(set.Logger, cf)

return exporterhelper.NewLogsExporter(
ctx,
set,
cfg,
exporter.exportLogs,
)
}
Loading
Loading