Skip to content

Commit

Permalink
[exporter/azuremonitor] Add Connection String Support to Azure Monito…
Browse files Browse the repository at this point in the history
…r Exporter (open-telemetry#28854)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
This pull request introduces the ability to configure the Azure Monitor
Exporter using a connection string, aligning the exporter configuration
with Azure Monitor's recommended practices. The current implementation
requires users to set the instrumentation key directly, which will soon
be deprecated in favor of using the connection string.

**Changes Made:**

1. Configuration Update: Modified the `Config` struct and related
configuration parsing logic to support a `ConnectionString` field.
2. Parsing Logic: Implemented functionality to parse the connection
string and extract necessary details, such as `InstrumentationKey` and
`IngestionEndpoint`.
3. Updated Tests: Revised existing tests and added new ones to ensure
coverage of the new configuration option.

**Benefits:**

* Streamlines the configuration process for end-users.
* Aligns with Azure Monitor's best practices and recommended
configuration approach.
* Paves the way for the upcoming deprecation of direct instrumentation
key configuration.

**Backwards Compatibility:**
This update maintains full backwards compatibility. Users currently
utilizing the instrumentation key for configuration can continue to do
so but are advised to transition to using the connection string.

**To-Do** 

* Documentation Update in a follow up PR
* Deprecation Notice: A future update will introduce a deprecation
warning for users still configuring the exporter with the
instrumentation key, encouraging them to switch to using a connection
string.
* Add support for `EndpointSuffix` in connection string -
https://learn.microsoft.com/en-us/azure/azure-monitor/app/sdk-connection-string?tabs=dotnet5#connection-string-with-an-endpoint-suffix

**Link to tracking Issue:** <Issue number if applicable> 
open-telemetry#28853

**Testing:** <Describe what testing was performed and which tests were
added.>

Conducted comprehensive testing, including unit tests, to validate that
the new configuration option works as expected and does not introduce
regressions. All tests are currently passing.

```
[Wed Nov  1 12:53:42 PDT 2023] --------- Transmitting 27 items ---------
[Wed Nov  1 12:53:43 PDT 2023] Telemetry transmitted in 331.926261ms
[Wed Nov  1 12:53:43 PDT 2023] Response: 200
[Wed Nov  1 12:53:43 PDT 2023] Items accepted/received: 27/27
[Wed Nov  1 12:53:53 PDT 2023] --------- Transmitting 30 items ---------
[Wed Nov  1 12:53:53 PDT 2023] Telemetry transmitted in 73.171392ms
[Wed Nov  1 12:53:53 PDT 2023] Response: 200
[Wed Nov  1 12:53:53 PDT 2023] Items accepted/received: 30/30
[Wed Nov  1 12:54:04 PDT 2023] --------- Transmitting 27 items ---------
[Wed Nov  1 12:54:04 PDT 2023] Telemetry transmitted in 68.037724ms
[Wed Nov  1 12:54:04 PDT 2023] Response: 200
[Wed Nov  1 12:54:04 PDT 2023] Items accepted/received: 27/27
```

**Documentation:** <Describe the documentation added.>

TODO, in a follow up PR.
  • Loading branch information
rajkumar-rangaraj authored and RoryCrispin committed Nov 24, 2023
1 parent 168d7c4 commit 4817905
Show file tree
Hide file tree
Showing 11 changed files with 307 additions and 13 deletions.
33 changes: 33 additions & 0 deletions .chloggen/add-connection-string-azure-monitor-exporter.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: azuremonitorexporter

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Added connection string support to the Azure Monitor Exporter

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [28853]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext: |
This enhancement simplifies the configuration process and aligns the exporter with Azure Monitor's recommended practices.
The Connection String method allows the inclusion of various fields such as the InstrumentationKey and IngestionEndpoint
within a single string, facilitating an easier and more integrated setup.
While the traditional InstrumentationKey method remains supported for backward compatibility, it will be phased out.
Users are encouraged to adopt the Connection String approach to ensure future compatibility and to leverage the broader
configuration options it enables.
# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: [user]
2 changes: 2 additions & 0 deletions cmd/otelcontribcol/exporters_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (
"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/config/configgrpc"
"go.opentelemetry.io/collector/config/configopaque"
"go.opentelemetry.io/collector/exporter"
"go.opentelemetry.io/collector/exporter/exportertest"
"go.opentelemetry.io/collector/exporter/otlpexporter"
Expand Down Expand Up @@ -298,6 +299,7 @@ func TestDefaultExporters(t *testing.T) {
getConfigFn: func() component.Config {
cfg := expFactories["azuremonitor"].CreateDefaultConfig().(*azuremonitorexporter.Config)
cfg.Endpoint = "http://" + endpoint
cfg.ConnectionString = configopaque.String("InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=" + cfg.Endpoint)

return cfg
},
Expand Down
2 changes: 1 addition & 1 deletion cmd/otelcontribcol/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ require (
github.com/stretchr/testify v1.8.4
go.opentelemetry.io/collector/component v0.88.1-0.20231026220224-6405e152a2d9
go.opentelemetry.io/collector/config/configgrpc v0.88.1-0.20231026220224-6405e152a2d9
go.opentelemetry.io/collector/config/configopaque v0.88.1-0.20231026220224-6405e152a2d9
go.opentelemetry.io/collector/confmap v0.88.1-0.20231026220224-6405e152a2d9
go.opentelemetry.io/collector/connector v0.88.1-0.20231026220224-6405e152a2d9
go.opentelemetry.io/collector/connector/forwardconnector v0.88.1-0.20231026220224-6405e152a2d9
Expand Down Expand Up @@ -637,7 +638,6 @@ require (
go.opentelemetry.io/collector/config/configcompression v0.88.1-0.20231026220224-6405e152a2d9 // indirect
go.opentelemetry.io/collector/config/confighttp v0.88.1-0.20231026220224-6405e152a2d9 // indirect
go.opentelemetry.io/collector/config/confignet v0.88.1-0.20231026220224-6405e152a2d9 // indirect
go.opentelemetry.io/collector/config/configopaque v0.88.1-0.20231026220224-6405e152a2d9 // indirect
go.opentelemetry.io/collector/config/configtelemetry v0.88.1-0.20231026220224-6405e152a2d9 // indirect
go.opentelemetry.io/collector/config/configtls v0.88.1-0.20231026220224-6405e152a2d9 // indirect
go.opentelemetry.io/collector/config/internal v0.88.1-0.20231026220224-6405e152a2d9 // indirect
Expand Down
23 changes: 20 additions & 3 deletions exporter/azuremonitorexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,37 @@ This exporter sends logs, traces and metrics to [Azure Monitor](https://docs.mic

## Configuration

The following settings are required:
To configure the Azure Monitor Exporter, you must specify one of the following settings:

- `instrumentation_key` (no default): Application Insights instrumentation key, which can be found in the Application Insights resource in the Azure Portal.
- `connection_string` (recommended): The Azure Application Insights Connection String is required to send telemetry data to the monitoring service. It is the recommended method for configuring the exporter, aligning with Azure Monitor's best practices. If you need guidance on creating Azure resources, please refer to the step-by-step guides to [Create an Application Insights resource](https://docs.microsoft.com/azure/azure-monitor/app/create-new-resource) and [find your connection string](https://docs.microsoft.com/azure/azure-monitor/app/sdk-connection-string?tabs=net#find-your-connection-string).
- `instrumentation_key`: Application Insights instrumentation key, which can be found in the Application Insights resource in the Azure Portal. While it is currently supported, its use is discouraged and it is slated for deprecation. It is highly encouraged to use the `connection_string` setting for new configurations and migrate existing configurations to use the `connection_string` as soon as possible.

**Important**: Only one of `connection_string` or `instrumentation_key` should be specified in your configuration. If both are provided, `connection_string` will be used as the priority setting.

The following settings can be optionally configured:

- `endpoint` (default = `https://dc.services.visualstudio.com/v2/track`): The endpoint URL where data will be submitted.
- `endpoint` (default = `https://dc.services.visualstudio.com/v2/track`): The endpoint URL where data will be submitted. While this option remains available, it is important to note that the use of the `connection_string` is recommended, as it encompasses the endpoint information. The direct configuration of the `endpoint` is considered to be on a deprecation path.
- `maxbatchsize` (default = 1024): The maximum number of telemetry items that can be submitted in each request. If this many items are buffered, the buffer will be flushed before `maxbatchinterval` expires.
- `maxbatchinterval` (default = 10s): The maximum time to wait before sending a batch of telemetry.
- `spaneventsenabled` (default = false): Enables export of span events.

Example:

```yaml
# Recommended Configuration:
# It is highly recommended to use the connection string which includes the InstrumentationKey and IngestionEndpoint
# This is the preferred method over using 'instrumentation_key' alone.
exporters:
azuremonitor:
connection_string: "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://ingestion.azuremonitor.com/"
```
(or)
```yaml
# Legacy Configuration:
# The use of 'instrumentation_key' alone is not recommended and will be deprecated in the future. It is advised to use the connection_string instead.
# This example is provided primarily for existing configurations that have not yet transitioned to the connection string.
exporters:
azuremonitor:
instrumentation_key: b1cd0778-85fc-4677-a3fa-79d3c23e0efd
Expand Down
1 change: 1 addition & 0 deletions exporter/azuremonitorexporter/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
// Config defines configuration for Azure Monitor
type Config struct {
Endpoint string `mapstructure:"endpoint"`
ConnectionString configopaque.String `mapstructure:"connection_string"`
InstrumentationKey configopaque.String `mapstructure:"instrumentation_key"`
MaxBatchSize int `mapstructure:"maxbatchsize"`
MaxBatchInterval time.Duration `mapstructure:"maxbatchinterval"`
Expand Down
3 changes: 2 additions & 1 deletion exporter/azuremonitorexporter/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ func TestLoadConfig(t *testing.T) {
id: component.NewIDWithName(metadata.Type, "2"),
expected: &Config{
Endpoint: defaultEndpoint,
InstrumentationKey: "abcdefg",
ConnectionString: "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://ingestion.azuremonitor.com/",
InstrumentationKey: "00000000-0000-0000-0000-000000000000",
MaxBatchSize: 100,
MaxBatchInterval: 10 * time.Second,
SpanEventsEnabled: false,
Expand Down
80 changes: 80 additions & 0 deletions exporter/azuremonitorexporter/connection_string_parser.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package azuremonitorexporter // import "github.com/open-telemetry/opentelemetry-collector-contrib/exporter/azuremonitorexporter"

import (
"fmt"
"net/url"
"path"
"strings"
)

type ConnectionVars struct {
InstrumentationKey string
IngestionURL string
}

const (
DefaultIngestionEndpoint = "https://dc.services.visualstudio.com/"
IngestionEndpointKey = "IngestionEndpoint"
InstrumentationKey = "InstrumentationKey"
ConnectionStringMaxLength = 4096
)

func parseConnectionString(exporterConfig *Config) (*ConnectionVars, error) {
connectionString := string(exporterConfig.ConnectionString)
instrumentationKey := string(exporterConfig.InstrumentationKey)
connectionVars := &ConnectionVars{}

if connectionString == "" && instrumentationKey == "" {
return nil, fmt.Errorf("ConnectionString and InstrumentationKey cannot be empty")
}
if len(connectionString) > ConnectionStringMaxLength {
return nil, fmt.Errorf("ConnectionString exceeds maximum length of %d characters", ConnectionStringMaxLength)
}
if connectionString == "" {
connectionVars.InstrumentationKey = instrumentationKey
connectionVars.IngestionURL = getIngestionURL(DefaultIngestionEndpoint)
return connectionVars, nil
}

pairs := strings.Split(connectionString, ";")
values := make(map[string]string)
for _, pair := range pairs {
kv := strings.SplitN(strings.TrimSpace(pair), "=", 2)
if len(kv) != 2 {
return nil, fmt.Errorf("invalid format for connection string: %s", pair)
}

key, value := strings.TrimSpace(kv[0]), strings.TrimSpace(kv[1])
if key == "" {
return nil, fmt.Errorf("key cannot be empty")
}
values[key] = value
}

var ok bool
if connectionVars.InstrumentationKey, ok = values[InstrumentationKey]; !ok || connectionVars.InstrumentationKey == "" {
return nil, fmt.Errorf("%s is required", InstrumentationKey)
}

var ingestionEndpoint string
if ingestionEndpoint, ok = values[IngestionEndpointKey]; !ok || ingestionEndpoint == "" {
ingestionEndpoint = DefaultIngestionEndpoint
}

connectionVars.IngestionURL = getIngestionURL(ingestionEndpoint)

return connectionVars, nil
}

func getIngestionURL(ingestionEndpoint string) string {
ingestionURL, err := url.Parse(ingestionEndpoint)
if err != nil {
ingestionURL, _ = url.Parse(DefaultIngestionEndpoint)
}

ingestionURL.Path = path.Join(ingestionURL.Path, "/v2/track")
return ingestionURL.String()
}
134 changes: 134 additions & 0 deletions exporter/azuremonitorexporter/connection_string_parser_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

package azuremonitorexporter

import (
"strings"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"go.opentelemetry.io/collector/config/configopaque"
)

func TestParseConnectionString(t *testing.T) {
tests := []struct {
name string
config *Config
want *ConnectionVars
wantError bool
}{
{
name: "Valid connection string and instrumentation key",
config: &Config{
ConnectionString: "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://ingestion.azuremonitor.com/",
InstrumentationKey: "00000000-0000-0000-0000-00000000IKEY",
},
want: &ConnectionVars{
InstrumentationKey: "00000000-0000-0000-0000-000000000000",
IngestionURL: "https://ingestion.azuremonitor.com/v2/track",
},
wantError: false,
},
{
name: "Empty connection string with valid instrumentation key",
config: &Config{
InstrumentationKey: "00000000-0000-0000-0000-000000000000",
},
want: &ConnectionVars{
InstrumentationKey: "00000000-0000-0000-0000-000000000000",
IngestionURL: DefaultIngestionEndpoint + "v2/track",
},
wantError: false,
},
{
name: "Valid connection string with empty instrumentation key",
config: &Config{
ConnectionString: "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://ingestion.azuremonitor.com/",
},
want: &ConnectionVars{
InstrumentationKey: "00000000-0000-0000-0000-000000000000",
IngestionURL: "https://ingestion.azuremonitor.com/v2/track",
},
wantError: false,
},
{
name: "Empty connection string and instrumentation key",
config: &Config{
ConnectionString: "",
InstrumentationKey: "",
},
want: nil,
wantError: true,
},
{
name: "Invalid connection string format",
config: &Config{
ConnectionString: "InvalidConnectionString",
},
want: nil,
wantError: true,
},
{
name: "Missing InstrumentationKey in connection string",
config: &Config{
ConnectionString: "IngestionEndpoint=https://ingestion.azuremonitor.com/",
},
want: nil,
wantError: true,
},
{
name: "Empty InstrumentationKey in connection string",
config: &Config{
ConnectionString: "InstrumentationKey=;IngestionEndpoint=https://ingestion.azuremonitor.com/",
},
want: nil,
wantError: true,
},
{
name: "Extra parameters in connection string",
config: &Config{
ConnectionString: "InstrumentationKey=00000000-0000-0000-0000-000000000000;IngestionEndpoint=https://ingestion.azuremonitor.com/;ExtraParam=extra",
},
want: &ConnectionVars{
InstrumentationKey: "00000000-0000-0000-0000-000000000000",
IngestionURL: "https://ingestion.azuremonitor.com/v2/track",
},
wantError: false,
},
{
name: "Spaces around equals in connection string",
config: &Config{
ConnectionString: "InstrumentationKey = 00000000-0000-0000-0000-000000000000 ; IngestionEndpoint = https://ingestion.azuremonitor.com/",
},
want: &ConnectionVars{
InstrumentationKey: "00000000-0000-0000-0000-000000000000",
IngestionURL: "https://ingestion.azuremonitor.com/v2/track",
},
wantError: false,
},
{
name: "Connection string too long",
config: &Config{
ConnectionString: configopaque.String(strings.Repeat("a", ConnectionStringMaxLength+1)),
},
want: nil,
wantError: true,
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := parseConnectionString(tt.config)
if tt.wantError {
require.Error(t, err, "Expected an error but got none")
} else {
require.NoError(t, err, "Unexpected error: %v", err)
require.NotNil(t, got, "Expected a non-nil result")
assert.Equal(t, tt.want.InstrumentationKey, got.InstrumentationKey, "InstrumentationKey does not match")
assert.Equal(t, tt.want.IngestionURL, got.IngestionURL, "IngestionEndpoint does not match")
}
})
}
}
Loading

0 comments on commit 4817905

Please sign in to comment.