Audit Compliance GDI specification pre-v1.7.0: Configuration

[pellared]

Review against https://github.com/signalfx/gdi-specification/blob/26087ae47670120998ae95ee33da2f3b63bf09bb/specification/configuration.md

[pellared]

One or more configuration variables may be needed to properly configure GDI repositories. Components that can be configured with environment variables MUST support configuration of these variables using environment variables.

✅ Configuration is pass-through to upstream, which respects the config env.

Any component that cannot be configured with environment variables MUST support configuration of these variables using an alternate method.

✅ Does manual configuration count?

Any component MAY support configuration of these variables by additional methods.

✅

GDI repositories MUST adopt stable and SHOULD adopt experimental configuration
variables in the OpenTelemetry Specification before proposing variables to the GDI specification.

✅ Nothing proposed at this time.

If a new configuration variable is needed by a GDI repository it MUST be brought to the GDI specification as a GitHub issue. The GDI specification maintainers SHOULD consider introducing needed configuration variables to the OpenTelemetry repository before approving Splunk-specific configuration variables.

✅

If a GDI repository requires an immediate configuration variable that is not available in the OpenTelemetry specification and not available in the GDI specification, the repository MAY introduce a repository-specific configuration variable until the GDI specification maintainers make a decision. Any repository-specific configuration variable defined SHOULD be prefixed with SPLUNK_ and MUST NOT be marked as stable unless added to the GDI specification. Upon resolution by the GDI specification, the GDI repository MUST change the repository-specific configuration variable by the repository's next minor release. This change MAY result in a breaking change so caution should be exhibited when considering repository-specific configuration variables.

✅

Splunk-specific configuration variables defined in the GDI specification MUST be prefixed with SPLUNK_. Furthermore, configuration specific to Splunk Observability Cloud MUST be prefixed with SPLUNK_OBSERVABILITY_ and to Splunk Enterprise or Splunk Cloud MUST be prefixed with SPLUNK_PLATFORM_.

✅

If a Splunk-specific configuration variable is declared as stable in the GDI specification and later the OpenTelemetry specification declares a similar variable as stable, the GDI specification MUST adopt the OpenTelemetry configuration variable and SHOULD mark the GDI specification configuration variable as deprecated by the next minor release. In addition to defining
Splunk-specification configuration variables, the GDI specification MAY require specific OpenTelemetry configuration variables be supported. If it does, the GDI specification MAY require certain values be supported including a specific default value.

✅

Whenever a configuration variable changes its name, a stable GDI repository (version >= 1.0) MUST support both old and new names until the next major release is done. The old configuration variable MUST NOT be removed in a minor release. GDI repositories that are not yet stable (version < 1.0) SHOULD follow this rule, but they are not required to. When it is detected that a user uses the old configuration variable a warning SHOULD be logged: the warning SHOULD state that the old variable is deprecated, the new one should be used instead, and that the old one will be removed in the next major release (not stable GDI repositories MAY remove deprecated features in any future release).

✅

Installation and configuration MUST optimize for customer experience and time-to-value. Installation and configuration MAY provide a mechanism for advanced or custom settings. As an example, the default installation for instrumentation libraries should provide everything needed to configure W3C and B3 as well as OTLP and Jaeger Thrift even though the default configuration is set to W3C and OTLP. Installing all of these dependencies by default may result in a large package, but makes it easy for users to switch settings via configuration. An advanced installation process can be provided where the user chooses what to install limiting the configuration options.

✅ In compliance except for Jaeger Thrift

For all use-cases that support environment variables (e.g. applications and serverless), it MUST be possible to configure an Instrumentation Library instance using the following environment variables:

Name	Default	Description
SPLUNK_ACCESS_TOKEN		Access token added to exported data. [1]
SPLUNK_PROFILER_CALL_STACK_INTERVAL	[7]	Interval at which call stacks are sampled (in ms) [5]
SPLUNK_PROFILER_ENABLED	false	Whether CPU profiling is enabled. [2] [5]
SPLUNK_PROFILER_LOGS_ENDPOINT	*	Where profiling data is sent. Defaults to the value in OTEL_EXPORTER_OTLP_ENDPOINT [5]
SPLUNK_PROFILER_MEMORY_ENABLED	false	Whether memory profiling is enabled. [2] [6]
SPLUNK_REALM	none	Which realm to send exported data. [3]
SPLUNK_TRACE_RESPONSE_HEADER_ENABLED	true	Whether Server-Timing header is added to HTTP responses. [4]

• [1]: Not user required if another system performs the authentication. For example, instrumentation libraries SHOULD send data to a locally running agent. The agent MAY define the access token that is used. If the instrumentation library is configured to send data directly to a Splunk back-end then this variable MUST be defined. Even when sending to an agent, instrumentation libraries MAY define this option (to support access_token_passthrough). This environment variable MUST work for otlp and jaeger-thrift-splunk exporters.
• [2]: The instrumentation library SHOULD NOT allow changing the setting at runtime, and the initial setting SHOULD be used for the entire lifespan of the application run. An instrumentation library whose profiling capability is deactivated MUST NOT introduce additional profiling-based overhead. It also MUST NOT emit profiling-based data.
• [3]: By default, instrumentation libraries are configured to send to a local collector (see OTEL_TRACES_EXPORTER below). If SPLUNK_REALM is set to anything besides none then the OTEL_EXPORTER_*_ENDPOINT is set to an endpoint based on the defined realm. If both SPLUNK_REALM and OTEL_EXPORTER_*_ENDPOINT are set then OTEL_EXPORTER_*_ENDPOINT takes precedence.
• [4]: If stitching of RUM spans and APM spans is desired then this parameter
  MUST be set to true.
• [5]: Applies only to instrumentation libraries with CPU profiling capabilities.
• [6]: Applies only to instrumentation libraries with memory profiling capabilities.
• [7]: 10000 for multi-threaded runtimes, 1000 for single-threaded runtimes.

✅

In addition to Splunk-specific environment variables, the following [OpenTelemetry environment
variables](https://github.com/open-telemetry/opentelemetry specification/blob/f228a83e652e5cd3ba96b9f780b704ee7a7daa4c/specification/sdk-environment-variables.md) are required.

• OTEL_SERVICE_NAME

  • Users MUST define a name for the service they are instrumenting. The service name can either be defined using the OTEL_SERVICE_NAME or OTEL_RESOURCE_ATTRIBUTES environment variable, or directly in their code. If the user fails to define a service name the distribution MUST log a warning. The warning message MUST clearly describe how to set the service name or link to relevant documentation. E.g.

    The service.name resource attribute is not set. Your service is unnamed and will be difficult to identify.
    Set your service name using the OTEL_SERVICE_NAME or OTEL_RESOURCE_ATTRIBUTES environment variable.
    E.g. `OTEL_SERVICE_NAME="<YOUR_SERVICE_NAME_HERE>"`

✅

• OTEL_PROPAGATORS
  • Distribution MUST default to "tracecontext,baggage"
  • Distribution MUST support and document how to switch to b3multi

X tracecontext,baggage are already default upstream. Will add doc.

• Span Collection Limits
  • Distribution MUST default to 1000 for OTEL_SPAN_LINK_COUNT_LIMIT (not OpenTelemetry default)
  • Distribution MUST default to 12000 for OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT (not OpenTelemetry default)
  • Distribution MUST default to unlimited for all others (not OpenTelemetry default)

✅

• OTEL_TRACES_EXPORTER

  • Non-RUM distribution MUST default to otlp using grpc or http/protobuf transport protocol.

  • Non-RUM distribution MAY offer jaeger-thrift-splunk that defaults to http://127.0.0.1:9080/v1/trace.
    NOTE: jaeger-thrift-splunk is deprecated. If the user selects jaeger-thrift-splunk, distributions MUST log a deprecation warning and suggest an alternate method. For example:

    jaeger-thrift-splunk trace exporter is deprecated and may be removed in a future major release. Use the default 
    OTLP exporter instead, or set the SPLUNK_REALM and SPLUNK_ACCESS_TOKEN environment variables to send 
    telemetry directly to Splunk Observability Cloud.

✅

• OTEL_TRACES_SAMPLER
  • Distribution MUST default to always_on
    (not OpenTelemetry default)

✅

In addition to environment variables, other ways of defining configuration also exist:

• Java System Properties: These properties MUST match the environment variables converting to lower case and replacing underscores with hyphens or periods. For example: system property splunk.trace-response-header.enabled is equivalent to environment variable SPLUNK_TRACE_RESPONSE_HEADER_ENABLED.

❓