graduate the new googlecloud exporter to beta (open-telemetry#9471)

djaglowski · May 10, 2022 · 94a314f · 94a314f
1 parent dfcd37f
commit 94a314f
Show file tree

Hide file tree

Showing 3 changed files with 64 additions and 63 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -32,6 +32,7 @@
 - `kafkametricsreceiver`: instrumentation name updated from `otelcol/kafkametrics` to `otelcol/kafkametricsreceiver` (#9406)
 - `kubeletstatsreceiver`: instrumentation name updated from `kubeletstats` to `otelcol/kubeletstatsreceiver` (#9400)
 - `datadogexporter`: Remove `GetHostTags` method from `TagsConfig` struct (#9423)
+- `googlecloudexporter`: Graduate the `exporter.googlecloud.OTLPDirect` feature-gate to Beta.  This includes changes to the configuration structure, and many changes to default behavior. (#9471)
 
 ### 🚩 Deprecations 🚩
 

diff --git a/exporter/googlecloudexporter/README.md b/exporter/googlecloudexporter/README.md
@@ -124,12 +124,7 @@ These instructions are to get you up and running quickly with the GCP exporter i
 The following configuration options are supported:
 
 - `project` (optional): GCP project identifier.
-- `endpoint` (optional): Endpoint where data is going to be sent to.
 - `user_agent` (optional): Override the user agent string sent on requests to Cloud Monitoring (currently only applies to metrics). Specify `{{version}}` to include the application version number. Defaults to `opentelemetry-collector-contrib {{version}}`.
-- `use_insecure` (optional): If true. use gRPC as their communication transport. Only has effect if Endpoint is not "".
-- `timeout` (optional): Timeout for all API calls. If not set, defaults to 12 seconds.
-- `resource_mappings` (optional): ResourceMapping defines mapping of resources from source (OpenCensus) to target (Google Cloud).
-  - `label_mappings` (optional): Optional flag signals whether we can proceed with transformation if a label is missing in the resource.
 - `retry_on_failure` (optional): Configuration for how to handle retries when sending data to Google Cloud fails.
   - `enabled` (default = true)
   - `initial_interval` (default = 5s): Time to wait after the first failure before retrying; ignored if `enabled` is `false`
@@ -147,8 +142,22 @@ Note: These `retry_on_failure` and `sending_queue` are provided (and documented)
 
 Additional configuration for the metric exporter:
 
-- `metric.prefix` (optional): MetricPrefix overrides the prefix / namespace of the Google Cloud metric type identifier. If not set, defaults to "custom.googleapis.com/opencensus/"
-- `metric.skip_create_descriptor` (optional): Whether to skip creating the metric descriptor.
+- `metric.endpoint` (optional): Endpoint where metric data is going to be sent to. Replaces `endpoint`.
+- `metric.use_insecure` (optional): If true, use gRPC as their communication transport. Only has effect if Endpoint is not "".
+- `metric.known_domains` (optional): If a metric belongs to one of these domains it does not get a prefix.
+- `metric.skip_create_descriptor` (optional): If set to true, do not send metric descriptors to GCM.
+- `metric.instrumentation_library_labels` (optional): If true, set the instrumentation_source and instrumentation_version labels. Defaults to true.
+- `metric.create_service_timeseries` (optional): If true, this will send all timeseries using `CreateServiceTimeSeries`. Implicitly, this sets `skip_create_descriptor` to true.
+- `metric.create_metric_descriptor_buffer_size` (optional): Buffer size for the channel which asynchronously calls CreateMetricDescriptor. Default is 10.
+- `metric.service_resource_labels` (optional):  If true, the exporter will copy OTel's service.name, service.namespace, and service.instance.id resource attributes into the GCM timeseries metric labels. Default is true.
+- `metric.resource_filters` (optional): If provided, resource attributes matching any filter will be included in metric labels. Defaults to empty, which won't include any additional resource labels.
+  - `prefix`: Match resource keys by prefix
+
+Additional configuration added for the trace exporter:
+
+- `trace.endpoint` (optional): Endpoint where trace data is going to be sent to. Replaces `endpoint`.
+- `trace.use_insecure` (optional): If true. use gRPC as their communication transport. Only has effect if Endpoint is not "". Replaces `use_insecure`.
+- `trace.attribute_mappings` (optional): AttributeMappings determines how to map from OpenTelemetry attribute keys to Google Cloud Trace keys.  By default, it changes http and service keys so that they appear more prominently in the UI.
 
 Example:
 
@@ -161,29 +170,31 @@ exporters:
     retry_on_failure:
       enabled: false
     project: my-project
-    endpoint: test-endpoint
     user_agent: my-collector {{version}}
-    use_insecure: true
-    timeout: 12s
-
-    resource_mappings:
-      - source_type: source.resource1
-        target_type: target-resource1
-        label_mappings:
-          - source_key: contrib.opencensus.io/exporter/googlecloud/project_id
-            target_key: project_id
-            optional: true
-          - source_key: source.label1
-            target_key: target_label_1
+    trace:
+      endpoint: cloudtrace.googleapis.com
+      use_insecure: false
+      attribute_mappings:
+      - key: service.name
+        replacement: g.co/gae/app/module
+    metric:
+      endpoint: monitoring.googleapis.com
+      use_insecure: false
+      prefix: workload.googleapis.com
+      skip_create_descriptor: false
+      known_domains:
+      - mycustomdomain.googleapis.com
+      instrumentation_library_labels: true
+      service_resource_labels: true
+      create_service_timeseries: false
+      create_metric_descriptor_buffer_size: 10
+      resource_filters:
+      - prefix: k8s.*
 
     sending_queue:
       enabled: true
       num_consumers: 2
       queue_size: 50
-
-    metric:
-      prefix: prefix
-      skip_create_descriptor: true
 ```
 
 Beyond standard YAML configuration as outlined in the sections that follow,
@@ -209,48 +220,37 @@ optimal network usage and avoiding memory overruns.  You may also want to run an
 
 See the [Collector feature gates](https://github.com/open-telemetry/opentelemetry-collector/blob/main/service/featuregate/README.md#collector-feature-gates) for an overview of feature gates in the collector.
 
-**ALPHA**: `exporter.googlecloud.OTLPDirect`
-
-When enabled via `--feature-gates=exporter.googlecloud.OTLPDirect`, the googlecloud exporter translates pdata directly to google cloud monitoring's types, rather than first translating to opencensus.  See the [Breaking Changes documentation](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/main/exporter/collector/breaking-changes.md#breaking-changes-vs-old-googlecloud-exporter) for breaking changes that will occur as a result of enabling this feature.
-
-Additional configuration added for the metric exporter:
-
-- `metric.endpoint` (optional): Endpoint where metric data is going to be sent to. Replaces `endpoint`.
-- `metric.use_insecure` (optional): If true. use gRPC as their communication transport. Only has effect if Endpoint is not "". Replaces `use_insecure`.
-- `metric.known_domains` (optional): If a metric belongs to one of these domains it does not get a prefix.
-- `metric.instrumentation_library_labels` (optional): If true, set the instrumentation_source and instrumentation_version labels. Defaults to true.
-- `metric.create_service_timeseries` (optional): If true, this will send all timeseries using `CreateServiceTimeSeries`. Implicitly, this sets `SkipMetricDescriptor` to true.
-- `metric.create_metric_descriptor_buffer_size` (optional): Buffer size for the channel which asynchronously calls CreateMetricDescriptor. Default is 10.
-- `metric.service_resource_labels` (optional):  If true, the exporter will copy OTel's service.name, service.namespace, and service.instance.id resource attributes into the GCM timeseries metric labels. Default is true.
-- `metric.resource_filters` (optional): If provided, resource attributes matching any filter will be included in metric labels. Defaults to empty, which won't include any additional resource labels.
-  - `prefix`: Match resource keys by prefix
-
-Additional configuration added for the trace exporter:
-
-- `trace.endpoint` (optional): Endpoint where trace data is going to be sent to. Replaces `endpoint`.
-- `trace.use_insecure` (optional): If true. use gRPC as their communication transport. Only has effect if Endpoint is not "". Replaces `use_insecure`.
-
-Removed configuration:
-
-- `endpoint`: replaced by `trace.endpoint` and `metric.endpoint`
-- `use_insecure`: replaced by `trace.use_insecure` and `metric.use_insecure`
-- `resource_mappings`: replacement is still under development
+**BETA**: `exporter.googlecloud.OTLPDirect`
 
-Changes to defaults:
+The `exporter.googlecloud.OTLPDirect` is enabled by default starting in v0.50.0, and can be disabled via `--feature-gates=-exporter.googlecloud.OTLPDirect`. The new googlecloud exporter translates pdata directly to google cloud monitoring's types, rather than first translating to opencensus.  See the [Breaking Changes documentation](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/main/exporter/collector/breaking-changes.md#breaking-changes-vs-old-googlecloud-exporter) for breaking changes that will occur as a result of this feature.
 
-- `prefix`: default is now `workload.googleapis.com`
+If you are broken by changes described there, or have encountered an issue with the new implementation, please open an issue [here](https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/issues/new)
 
-Additional Behavior:
+If you disable the feature-gate, you can continue to set the legacy configuration options:
 
-- Metrics are now batched in the exporter to handle edge cases around summar metrics.  The batch processor is no longer needed in metrics processors.
+- `project` (optional): GCP project identifier.
+- `endpoint` (optional): Endpoint where data is going to be sent to.
+- `user_agent` (optional): Override the user agent string sent on requests to Cloud Monitoring (currently only applies to metrics). Specify `{{version}}` to include the application version number. Defaults to `opentelemetry-collector-contrib {{version}}`.
+- `use_insecure` (optional): If true. use gRPC as their communication transport. Only has effect if Endpoint is not "".
+- `timeout` (optional): Timeout for all API calls. If not set, defaults to 12 seconds.
+- `resource_mappings` (optional): ResourceMapping defines mapping of resources from source (OpenCensus) to target (Google Cloud).
+  - `label_mappings` (optional): Optional flag signals whether we can proceed with transformation if a label is missing in the resource.
+- `retry_on_failure` (optional): Configuration for how to handle retries when sending data to Google Cloud fails.
+  - `enabled` (default = true)
+  - `initial_interval` (default = 5s): Time to wait after the first failure before retrying; ignored if `enabled` is `false`
+  - `max_interval` (default = 30s): Is the upper bound on backoff; ignored if `enabled` is `false`
+  - `max_elapsed_time` (default = 120s): Is the maximum amount of time spent trying to send a batch; ignored if `enabled` is `false`
+- `sending_queue` (optional): Configuration for how to buffer traces before sending.
+  - `enabled` (default = true)
+  - `num_consumers` (default = 10): Number of consumers that dequeue batches; ignored if `enabled` is `false`
+  - `queue_size` (default = 5000): Maximum number of batches kept in memory before data; ignored if `enabled` is `false`;
+    User should calculate this as `num_seconds * requests_per_second` where:
+    - `num_seconds` is the number of seconds to buffer in case of a backend outage
+    - `requests_per_second` is the average number of requests per seconds.
 
-## Deprecatations
+Note: These `retry_on_failure` and `sending_queue` are provided (and documented) by the [Exporter Helper](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter/exporterhelper#configuration)
 
-The previous trace configuration (v0.21.0) has been deprecated in favor of the common configuration options available in OpenTelemetry. These will cause a failure to start
-and should be migrated:
+Additional configuration for the metric exporter:
 
-- `trace.bundle_delay_threshold` (optional): Use `batch` processor instead ([docs](https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/batchprocessor)).
-- `trace.bundle_count_threshold` (optional): Use `batch` processor instead ([docs](https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/batchprocessor)).
-- `trace.bundle_byte_threshold` (optional): Use `memorylimiter` processor instead ([docs](https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/memorylimiterprocessor))
-- `trace.bundle_byte_limit` (optional): Use `memorylimiter` processor instead ([docs](https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/memorylimiterprocessor))
-- `trace.buffer_max_bytes` (optional): Use `memorylimiter` processor instead ([docs](https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/memorylimiterprocessor))
+- `metric.prefix` (optional): MetricPrefix overrides the prefix / namespace of the Google Cloud metric type identifier. If not set, defaults to "custom.googleapis.com/opencensus/"
+- `metric.skip_create_descriptor` (optional): Whether to skip creating the metric descriptor.
diff --git a/exporter/googlecloudexporter/factory.go b/exporter/googlecloudexporter/factory.go
@@ -36,7 +36,7 @@ func init() {
 	featuregate.GetRegistry().MustRegister(featuregate.Gate{
 		ID:          pdataExporterFeatureGate,
 		Description: "When enabled, the googlecloud exporter translates pdata directly to google cloud monitoring's types, rather than first translating to opencensus.",
-		Enabled:     false,
+		Enabled:     true,
 	})
 }