Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FE Release 2024-09-16 #3129

Merged
merged 16 commits into from
Sep 17, 2024
Merged
Changes from 12 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -102,6 +102,8 @@ linters:
- perfsprint
# malfunctions on embedded structs
- typecheck
# magic numbers
- mnd
Comment on lines +105 to +106
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enable the mnd linter to detect magic numbers.

The mnd linter is being added to the list of disabled linters, which contradicts the purpose of introducing it. To effectively detect and prevent the use of magic numbers in the codebase, the mnd linter should be enabled.

Remove the following lines to enable the mnd linter:

-    # magic numbers
-    - mnd
Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# magic numbers
- mnd

fast: false

issues:

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion agents/contracts/destination/destination.metadata.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion agents/contracts/gasoracle/gasoracle.metadata.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion agents/contracts/inbox/inbox.metadata.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion agents/contracts/lightinbox/lightinbox.metadata.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion agents/contracts/lightmanager/lightmanager.metadata.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion agents/contracts/origin/origin.metadata.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion agents/contracts/summit/summit.metadata.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion agents/contracts/test/gasdata/gasdataharness.metadata.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion agents/contracts/test/testclient/testclient.metadata.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion core/go.mod
Original file line number Diff line number Diff line change
@@ -63,6 +63,7 @@ require (
go.uber.org/multierr v1.11.0
go.uber.org/zap v1.27.0
golang.org/x/sync v0.8.0
google.golang.org/grpc v1.64.0
gorm.io/driver/sqlite v1.5.6
gorm.io/gorm v1.25.10
k8s.io/apimachinery v0.29.3
@@ -189,7 +190,6 @@ require (
golang.org/x/tools v0.24.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240701130421-f6361c86f094 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240701130421-f6361c86f094 // indirect
google.golang.org/grpc v1.64.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
22 changes: 21 additions & 1 deletion core/metrics/README.md
Original file line number Diff line number Diff line change
@@ -12,7 +12,27 @@ There's also a `NAME_PREFIX` environment variable that will prefix all the metri

## OTLP

We do our best to support enviornment variables specified in the [Otel Spec](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/) and have added a few of our own. Key ones to note are:
We do our best to support enviornment variables specified in the [Otel Spec](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/) and the [OTLP Spec](https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/) and have added a few of our own. This was to allow for multiple exporter backends for traces, as otel clients only allow for one URL. The relevant multi exporter code is in `multiexporter.go`, and simply wraps multiple otel clients.

The additional environment variables to note are:
| Enviornment Variable | Description | Default |
|------------------------------------------|-------------------------------------------|---------|
| `OTEL_EXPORTER_OTLP_ENDPOINT` | The endpoint for the primary OTLP exporter | None |
| `OTEL_EXPORTER_OTLP_ENDPOINT_1` | The endpoint for the first additional OTLP exporter | None |
| `OTEL_EXPORTER_OTLP_ENDPOINT_2` | The endpoint for the second additional OTLP exporter | None |
| `OTEL_EXPORTER_OTLP_ENDPOINT_3` | The endpoint for the third additional OTLP exporter | None |
| ... | Additional endpoints can be added by incrementing the number | None |
| `OTEL_EXPORTER_OTLP_TRANSPORT` | The transport protocol for the primary OTLP exporter | `http` |
| `OTEL_EXPORTER_OTLP_TRANSPORT_1` | The transport protocol for the first additional OTLP exporter | `http` |
| `OTEL_EXPORTER_OTLP_TRANSPORT_2` | The transport protocol for the second additional OTLP exporter | `http` |
| `OTEL_EXPORTER_OTLP_TRANSPORT_3` | The transport protocol for the third additional OTLP exporter | `http` |
| ... | Additional transports can be specified by incrementing the number | `http` |

You can do the same thing for `OTEL_EXPORTER_OTLP_SECURE_MODE` and `OTEL_EXPORTER_OTLP_HEADERS`

<!-- TODO: fully document these optins-->

Note: The OTLP exporter endpoints and transports can be specified for multiple exporters by using incrementing numbers (1, 2, 3, etc.) in the environment variable names. This allows for configuration of multiple OTLP exporters. The primary exporter uses the base names without numbers.


## Jaeger
6 changes: 6 additions & 0 deletions core/metrics/export_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package metrics

// HeadersToMap converts a string of headers to a map.
func HeadersToMap(val string) map[string]string {
return headersToMap(val)
}
83 changes: 83 additions & 0 deletions core/metrics/multiexporter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
package metrics

import (
"context"
"fmt"
"go.uber.org/multierr"
"sync"
"time"

"go.opentelemetry.io/otel/sdk/trace"
)

// MultiExporter is an interface that allows exporting spans to multiple OTLP trace exporters.
type MultiExporter interface {
trace.SpanExporter
AddExporter(exporter trace.SpanExporter)
}

type multiExporter struct {
exporters []trace.SpanExporter
}

// NewMultiExporter creates a new multi exporter that forwards spans to multiple OTLP trace exporters.
// It takes in one or more trace.SpanExporter instances and ensures that spans are sent to all of them.
// This is useful when you need to send trace data to multiple backends or endpoints.
func NewMultiExporter(exporters ...trace.SpanExporter) MultiExporter {
return &multiExporter{
exporters: exporters,
}
}

const defaultTimeout = 30 * time.Second

// ExportSpans exports a batch of spans.
func (m *multiExporter) ExportSpans(parentCtx context.Context, ss []trace.ReadOnlySpan) error {
return m.doParallel(parentCtx, func(ctx context.Context, exporter trace.SpanExporter) error {
return exporter.ExportSpans(ctx, ss)
})
}

func (m *multiExporter) doParallel(parentCtx context.Context, fn func(context.Context, trace.SpanExporter) error) error {
ctx, cancel := context.WithTimeout(parentCtx, defaultTimeout)
defer cancel()

var wg sync.WaitGroup
var errors []error
var mu sync.Mutex

wg.Add(len(m.exporters))
for _, exporter := range m.exporters {
go func(exporter trace.SpanExporter) {
defer wg.Done()
err := fn(ctx, exporter)
if err != nil {
mu.Lock()
errors = append(errors, fmt.Errorf("error in doMultiple: %w", err))
mu.Unlock()
}
}(exporter)
}

wg.Wait()
if len(errors) > 0 {
// nolint: wrapcheck
return multierr.Combine(errors...)
}

return nil
}
Comment on lines +32 to +69
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests to cover the missing lines in the doParallel method.

The implementation of the ExportSpans method and the doParallel helper method is correct and handles concurrency and error collection effectively.

However, static analysis hints indicate that some lines in the doParallel method are not covered by tests:

  • Lines 55-58: Appending errors to the errors slice.
  • Lines 64-66: Combining and returning the collected errors.

Please add tests to cover these lines and ensure the behavior is thoroughly tested.

Do you want me to generate the missing test cases or open a GitHub issue to track this task?

Tools
GitHub Check: codecov/patch

[warning] 55-58: core/metrics/multiexporter.go#L55-L58
Added lines #L55 - L58 were not covered by tests


[warning] 64-66: core/metrics/multiexporter.go#L64-L66
Added lines #L64 - L66 were not covered by tests


// Shutdown notifies the exporter of a pending halt to operations.
func (m *multiExporter) Shutdown(ctx context.Context) error {
return m.doParallel(ctx, func(ctx context.Context, exporter trace.SpanExporter) error {
return exporter.Shutdown(ctx)
})
}

// AddExporter adds an exporter to the multi exporter.
func (m *multiExporter) AddExporter(exporter trace.SpanExporter) {
m.exporters = append(m.exporters, exporter)
}

var _ trace.SpanExporter = &multiExporter{}
Comment on lines +78 to +83
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests to cover the AddExporter method.

The implementation of the AddExporter method is correct and adds a new exporter to the exporters slice. The type assertion ensures that the multiExporter struct implements the trace.SpanExporter interface.

However, static analysis hints indicate that the lines in the AddExporter method are not covered by tests:

  • Lines 79-80: Appending the new exporter to the exporters slice.

Please add tests to cover the AddExporter method and ensure its behavior is thoroughly tested.

Do you want me to generate the missing test cases or open a GitHub issue to track this task?

Tools
GitHub Check: codecov/patch

[warning] 79-80: core/metrics/multiexporter.go#L79-L80
Added lines #L79 - L80 were not covered by tests

Loading