[receiver/elasticsearchreceiver] emit network metrics with or without…

… a direction attribute (#13258) Add feature gates to the elastic search receiver so that it can emit network metrics with or without a direction attribute.
open-telemetry · Aug 12, 2022 · 73b3ae5 · 73b3ae5
1 parent c3aa49a
commit 73b3ae5
Show file tree

Hide file tree

Showing 8 changed files with 1,861 additions and 11 deletions.
diff --git a/receiver/elasticsearchreceiver/README.md b/receiver/elasticsearchreceiver/README.md
@@ -48,5 +48,45 @@ The full list of settings exposed for this receiver are documented [here](./conf
 
 Details about the metrics produced by this receiver can be found in [metadata.yaml](./metadata.yaml)
 
+### Feature gate configurations
+
+#### Transition from metrics with "direction" attribute
+
+Some elasticsearch metrics reported are transitioning from being reported with a `direction` attribute to being reported with the
+direction included in the metric name to adhere to the OpenTelemetry specification
+(https://github.com/open-telemetry/opentelemetry-specification/pull/2617):
+
+- `elasticsearch.node.cluster.io` will become:
+  - `elasticsearch.node.cluster.io.received`
+  - `elasticsearch.node.cluster.io.sent`
+
+The following feature gates control the transition process:
+
+- **receiver.elasticsearchreceiver.emitMetricsWithoutDirectionAttribute**: controls if the new metrics without `direction` attribute are emitted by the receiver.
+- **receiver.elasticsearchreceiver.emitMetricsWithDirectionAttribute**: controls if the deprecated metrics with `direction` attribute are emitted by the receiver.
+
+##### Transition schedule:
+
+See this [tracking issue](https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/11815) for more details.
+
+1. Phase 1, v0.59.0, August 2022:
+
+- The new metrics are available for all scrapers, but disabled by default, they can be enabled with the feature gates.
+- `receiver.elasticsearchreceiver.emitMetricsWithDirectionAttribute` is enabled by default.
+- `receiver.elasticsearchreceiver.emitMetricsWithoutDirectionAttribute` is disabled by default.
+
+2. Phase 2, version and date TBD:
+
+- The new metrics are enabled by default, deprecated metrics disabled, they can be enabled with the feature gates.
+- `receiver.elasticsearchreceiver.emitMetricsWithDirectionAttribute` is disabled by default.
+- `receiver.elasticsearchreceiver.emitMetricsWithoutDirectionAttribute` is enabled by default.
+
+3. Phase 3, version and date TBD:
+
+- The feature gates are removed.
+- The new metrics without `direction` attribute are always emitted.
+- The deprecated metrics with `direction` attribute are no longer available.
+
+
 [beta]:https://github.com/open-telemetry/opentelemetry-collector#beta
 [contrib]:https://github.com/open-telemetry/opentelemetry-collector-releases/tree/main/distributions/otelcol-contrib
diff --git a/receiver/elasticsearchreceiver/documentation.md b/receiver/elasticsearchreceiver/documentation.md
@@ -19,6 +19,8 @@ These are the metrics available for this scraper.
 | **elasticsearch.node.cache.memory.usage** | The size in bytes of the cache. | By | Sum(Int) | <ul> <li>cache_name</li> </ul> |
 | **elasticsearch.node.cluster.connections** | The number of open tcp connections for internal cluster communication. | {connections} | Sum(Int) | <ul> </ul> |
 | **elasticsearch.node.cluster.io** | The number of bytes sent and received on the network for internal cluster communication. | By | Sum(Int) | <ul> <li>direction</li> </ul> |
+| **elasticsearch.node.cluster.io.received** | The number of bytes received on the network for internal cluster communication. | By | Sum(Int) | <ul> </ul> |
+| **elasticsearch.node.cluster.io.sent** | The number of bytes sent on the network for internal cluster communication. | By | Sum(Int) | <ul> </ul> |
 | **elasticsearch.node.disk.io.read** | The total number of kilobytes read across all file stores for this node. | By | Sum(Int) | <ul> </ul> |
 | **elasticsearch.node.disk.io.write** | The total number of kilobytes written across all file stores for this node. | By | Sum(Int) | <ul> </ul> |
 | **elasticsearch.node.documents** | The number of documents on the node. | {documents} | Sum(Int) | <ul> <li>document_state</li> </ul> |

diff --git a/receiver/elasticsearchreceiver/internal/metadata/generated_metrics_v2.go b/receiver/elasticsearchreceiver/internal/metadata/generated_metrics_v2.go
diff --git a/receiver/elasticsearchreceiver/metadata.yaml b/receiver/elasticsearchreceiver/metadata.yaml
@@ -162,6 +162,7 @@ metrics:
       value_type: int
     attributes: [ ]
     enabled: true
+  # produced when receiver.elasticsearchreceiver.emitMetricsWithDirectionAttribute feature gate is enabled
   elasticsearch.node.cluster.io:
     description: The number of bytes sent and received on the network for internal cluster communication.
     unit: By
@@ -171,6 +172,26 @@ metrics:
       value_type: int
     attributes: [direction]
     enabled: true
+  # produced when receiver.elasticsearchreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled
+  elasticsearch.node.cluster.io.received:
+    description: The number of bytes received on the network for internal cluster communication.
+    unit: By
+    sum:
+      monotonic: true
+      aggregation: cumulative
+      value_type: int
+    attributes: []
+    enabled: true
+  # produced when receiver.elasticsearchreceiver.emitMetricsWithoutDirectionAttribute feature gate is enabled
+  elasticsearch.node.cluster.io.sent:
+    description: The number of bytes sent on the network for internal cluster communication.
+    unit: By
+    sum:
+      monotonic: true
+      aggregation: cumulative
+      value_type: int
+    attributes: []
+    enabled: true
   elasticsearch.node.cluster.connections:
     description: The number of open tcp connections for internal cluster communication.
     unit: "{connections}"
@@ -339,7 +360,7 @@ metrics:
     gauge:
       value_type: int
     attributes: []
-    enabled: true  
+    enabled: true
   jvm.memory.heap.used:
     description: The current heap memory usage
     unit: By
@@ -419,7 +440,7 @@ metrics:
     enabled: true
   elasticsearch.cluster.health:
     description: The health status of the cluster.
-    extended_documentation: 
+    extended_documentation:
       Health status is based on the state of its primary and replica shards.
       Green indicates all shards are assigned.
       Yellow indicates that one or more replica shards are unassigned.

diff --git a/receiver/elasticsearchreceiver/scraper.go b/receiver/elasticsearchreceiver/scraper.go
@@ -24,27 +24,64 @@ import (
 	"go.opentelemetry.io/collector/pdata/pcommon"
 	"go.opentelemetry.io/collector/pdata/pmetric"
 	"go.opentelemetry.io/collector/receiver/scrapererror"
+	"go.opentelemetry.io/collector/service/featuregate"
 
 	"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/elasticsearchreceiver/internal/metadata"
 )
 
+const (
+	emitMetricsWithDirectionAttributeFeatureGateID    = "receiver.elasticsearchreceiver.emitMetricsWithDirectionAttribute"
+	emitMetricsWithoutDirectionAttributeFeatureGateID = "receiver.elasticsearchreceiver.emitMetricsWithoutDirectionAttribute"
+)
+
+var (
+	emitMetricsWithDirectionAttributeFeatureGate = featuregate.Gate{
+		ID:      emitMetricsWithDirectionAttributeFeatureGateID,
+		Enabled: true,
+		Description: "Some elasticsearch metrics reported are transitioning from being reported with a direction " +
+			"attribute to being reported with the direction included in the metric name to adhere to the " +
+			"OpenTelemetry specification. This feature gate controls emitting the old metrics with the direction " +
+			"attribute. For more details, see: " +
+			"https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/elasticsearchreceiver/README.md#feature-gate-configurations",
+	}
+
+	emitMetricsWithoutDirectionAttributeFeatureGate = featuregate.Gate{
+		ID:      emitMetricsWithoutDirectionAttributeFeatureGateID,
+		Enabled: false,
+		Description: "Some elasticsearch metrics reported are transitioning from being reported with a direction " +
+			"attribute to being reported with the direction included in the metric name to adhere to the " +
+			"OpenTelemetry specification. This feature gate controls emitting the new metrics without the direction " +
+			"attribute. For more details, see: " +
+			"https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/elasticsearchreceiver/README.md#feature-gate-configurations",
+	}
+)
+
+func init() {
+	featuregate.GetRegistry().MustRegister(emitMetricsWithDirectionAttributeFeatureGate)
+	featuregate.GetRegistry().MustRegister(emitMetricsWithoutDirectionAttributeFeatureGate)
+}
+
 var errUnknownClusterStatus = errors.New("unknown cluster status")
 
 type elasticsearchScraper struct {
-	client   elasticsearchClient
-	settings component.TelemetrySettings
-	cfg      *Config
-	mb       *metadata.MetricsBuilder
+	client                               elasticsearchClient
+	settings                             component.TelemetrySettings
+	cfg                                  *Config
+	mb                                   *metadata.MetricsBuilder
+	emitMetricsWithDirectionAttribute    bool
+	emitMetricsWithoutDirectionAttribute bool
 }
 
 func newElasticSearchScraper(
 	settings component.ReceiverCreateSettings,
 	cfg *Config,
 ) *elasticsearchScraper {
 	return &elasticsearchScraper{
-		settings: settings.TelemetrySettings,
-		cfg:      cfg,
-		mb:       metadata.NewMetricsBuilder(cfg.Metrics, settings.BuildInfo),
+		settings:                             settings.TelemetrySettings,
+		cfg:                                  cfg,
+		mb:                                   metadata.NewMetricsBuilder(cfg.Metrics, settings.BuildInfo),
+		emitMetricsWithDirectionAttribute:    featuregate.GetRegistry().IsEnabled(emitMetricsWithDirectionAttributeFeatureGateID),
+		emitMetricsWithoutDirectionAttribute: featuregate.GetRegistry().IsEnabled(emitMetricsWithoutDirectionAttributeFeatureGateID),
 	}
 }
 
@@ -88,8 +125,15 @@ func (r *elasticsearchScraper) scrapeNodeMetrics(ctx context.Context, now pcommo
 		r.mb.RecordElasticsearchNodeDiskIoReadDataPoint(now, info.FS.IOStats.Total.ReadBytes)
 		r.mb.RecordElasticsearchNodeDiskIoWriteDataPoint(now, info.FS.IOStats.Total.WriteBytes)
 
-		r.mb.RecordElasticsearchNodeClusterIoDataPoint(now, info.TransportStats.ReceivedBytes, metadata.AttributeDirectionReceived)
-		r.mb.RecordElasticsearchNodeClusterIoDataPoint(now, info.TransportStats.SentBytes, metadata.AttributeDirectionSent)
+		if r.emitMetricsWithDirectionAttribute {
+			r.mb.RecordElasticsearchNodeClusterIoDataPoint(now, info.TransportStats.ReceivedBytes, metadata.AttributeDirectionReceived)
+			r.mb.RecordElasticsearchNodeClusterIoDataPoint(now, info.TransportStats.SentBytes, metadata.AttributeDirectionSent)
+		}
+
+		if r.emitMetricsWithoutDirectionAttribute {
+			r.mb.RecordElasticsearchNodeClusterIoReceivedDataPoint(now, info.TransportStats.ReceivedBytes)
+			r.mb.RecordElasticsearchNodeClusterIoSentDataPoint(now, info.TransportStats.SentBytes)
+		}
 
 		r.mb.RecordElasticsearchNodeClusterConnectionsDataPoint(now, info.TransportStats.OpenConnections)