Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metricbeat GCP] Rename stackdriver metricset to metrics #19718

Merged
merged 9 commits into from
Jul 14, 2020
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Add Tomcat overview dashboard {pull}14026[14026]
- Move service config under metrics and simplify metric types. {pull}18691[18691]
- Fix ECS compliance of user.id field in system/users metricset {pull}19019[19019]
- Rename googlecloud stackdriver metricset to metrics. {pull}19718[19718]

*Packetbeat*

Expand Down
4 changes: 2 additions & 2 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -18234,10 +18234,10 @@ type: object

--

*`googlecloud.stackdriver.*.*.*.*`*::
*`googlecloud.metrics.*.*.*.*`*::
+
--
Metrics that returned from StackDriver API query.
Metrics that returned from Google Cloud API query.


type: object
Expand Down
21 changes: 11 additions & 10 deletions metricbeat/docs/modules/googlecloud.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ For example, if Compute Metricset fetches 14 metrics (which is the number of met

[float]
== Metricsets
Currently, we have `compute`, `loadbalancing`, `pubsub`, `stackdriver` and
Currently, we have `compute`, `loadbalancing`, `pubsub`, `metrics` and
`storage` metricset in `googlecloud` module.

[float]
Expand Down Expand Up @@ -196,10 +196,11 @@ The `pubsub` metricset comes with a predefined dashboard:
image::./images/metricbeat-googlecloud-pubsub-overview.png[]

[float]
=== `stackdriver`
Stackdriver provides visibility into the performance, uptime, and overall health
of cloud-powered applications. It collects metrics, events, and metadata from
different services from Google Cloud.
=== `metrics`
`metrics` metricset uses Google Cloud Operations/Stackdriver, which provides
visibility into the performance, uptime, and overall health of cloud-powered
applications. It collects metrics, events, and metadata from different services
from Google Cloud.
This metricset is to collect https://cloud.google.com/monitoring/api/metrics_gcp[monitoring metrics]
from Google Cloud using `ListTimeSeries` API.

Expand Down Expand Up @@ -256,7 +257,7 @@ metricbeat.modules:

- module: googlecloud
metricsets:
- stackdriver
- metrics
project_id: "your project id"
credentials_file_path: "your JSON credentials file path"
exclude_labels: false
Expand All @@ -280,19 +281,19 @@ The following metricsets are available:

* <<metricbeat-metricset-googlecloud-loadbalancing,loadbalancing>>

* <<metricbeat-metricset-googlecloud-pubsub,pubsub>>
* <<metricbeat-metricset-googlecloud-metrics,metrics>>

* <<metricbeat-metricset-googlecloud-stackdriver,stackdriver>>
* <<metricbeat-metricset-googlecloud-pubsub,pubsub>>

* <<metricbeat-metricset-googlecloud-storage,storage>>

include::googlecloud/compute.asciidoc[]

include::googlecloud/loadbalancing.asciidoc[]

include::googlecloud/pubsub.asciidoc[]
include::googlecloud/metrics.asciidoc[]

include::googlecloud/stackdriver.asciidoc[]
include::googlecloud/pubsub.asciidoc[]

include::googlecloud/storage.asciidoc[]

Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
This file is generated! See scripts/mage/docs_collector.go
////

[[metricbeat-metricset-googlecloud-stackdriver]]
=== Google Cloud Platform stackdriver metricset
[[metricbeat-metricset-googlecloud-metrics]]
=== Google Cloud Platform metrics metricset

beta[]

include::../../../../x-pack/metricbeat/module/googlecloud/stackdriver/_meta/docs.asciidoc[]
include::../../../../x-pack/metricbeat/module/googlecloud/metrics/_meta/docs.asciidoc[]


==== Fields
Expand All @@ -19,5 +19,5 @@ Here is an example document generated by this metricset:

[source,json]
----
include::../../../../x-pack/metricbeat/module/googlecloud/stackdriver/_meta/data.json[]
include::../../../../x-pack/metricbeat/module/googlecloud/metrics/_meta/data.json[]
----
2 changes: 1 addition & 1 deletion metricbeat/docs/modules_list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,8 @@ This file is generated! See scripts/mage/docs_collector.go
|<<metricbeat-module-googlecloud,Google Cloud Platform>> beta[] |image:./images/icon-yes.png[Prebuilt dashboards are available] |
.5+| .5+| |<<metricbeat-metricset-googlecloud-compute,compute>> beta[]
|<<metricbeat-metricset-googlecloud-loadbalancing,loadbalancing>> beta[]
|<<metricbeat-metricset-googlecloud-metrics,metrics>> beta[]
|<<metricbeat-metricset-googlecloud-pubsub,pubsub>> beta[]
|<<metricbeat-metricset-googlecloud-stackdriver,stackdriver>> beta[]
|<<metricbeat-metricset-googlecloud-storage,storage>> beta[]
|<<metricbeat-module-graphite,Graphite>> |image:./images/icon-no.png[No prebuilt dashboards] |
.1+| .1+| |<<metricbeat-metricset-graphite-server,server>>
Expand Down
2 changes: 1 addition & 1 deletion x-pack/metricbeat/include/list.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion x-pack/metricbeat/metricbeat.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -526,7 +526,7 @@ metricbeat.modules:

- module: googlecloud
metricsets:
- stackdriver
- metrics
project_id: "your project id"
credentials_file_path: "your JSON credentials file path"
exclude_labels: false
Expand Down
2 changes: 1 addition & 1 deletion x-pack/metricbeat/module/googlecloud/_meta/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@

- module: googlecloud
metricsets:
- stackdriver
- metrics
project_id: "your project id"
credentials_file_path: "your JSON credentials file path"
exclude_labels: false
Expand Down
11 changes: 6 additions & 5 deletions x-pack/metricbeat/module/googlecloud/_meta/docs.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ For example, if Compute Metricset fetches 14 metrics (which is the number of met

[float]
== Metricsets
Currently, we have `compute`, `loadbalancing`, `pubsub`, `stackdriver` and
Currently, we have `compute`, `loadbalancing`, `pubsub`, `metrics` and
`storage` metricset in `googlecloud` module.

[float]
Expand Down Expand Up @@ -186,10 +186,11 @@ The `pubsub` metricset comes with a predefined dashboard:
image::./images/metricbeat-googlecloud-pubsub-overview.png[]

[float]
=== `stackdriver`
Stackdriver provides visibility into the performance, uptime, and overall health
of cloud-powered applications. It collects metrics, events, and metadata from
different services from Google Cloud.
=== `metrics`
`metrics` metricset uses Google Cloud Operations/Stackdriver, which provides
visibility into the performance, uptime, and overall health of cloud-powered
applications. It collects metrics, events, and metadata from different services
from Google Cloud.
This metricset is to collect https://cloud.google.com/monitoring/api/metrics_gcp[monitoring metrics]
from Google Cloud using `ListTimeSeries` API.

Expand Down
4 changes: 2 additions & 2 deletions x-pack/metricbeat/module/googlecloud/_meta/fields.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@
- name: system.*
type: object
object_type: keyword
- name: "stackdriver.*.*.*.*"
- name: "metrics.*.*.*.*"
type: object
object_type: double
object_type_mapping_type: "*"
description: >
Metrics that returned from StackDriver API query.
Metrics that returned from Google Cloud API query.
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import (

"github.com/elastic/beats/v7/libbeat/common"
mbtest "github.com/elastic/beats/v7/metricbeat/mb/testing"
"github.com/elastic/beats/v7/x-pack/metricbeat/module/googlecloud/stackdriver"
"github.com/elastic/beats/v7/x-pack/metricbeat/module/googlecloud/metrics"
)

func TestData(t *testing.T) {
Expand All @@ -36,7 +36,7 @@ func TestData(t *testing.T) {
{"googlecloud.compute.instance.memory", "./_meta/data_memory.json"},
}

config := stackdriver.GetConfigForTest(t, "compute")
config := metrics.GetConfigForTest(t, "compute")

for _, df := range dataFiles {
metricSet := mbtest.NewFetcher(t, config)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import (

// Register input module and metricset
_ "github.com/elastic/beats/v7/x-pack/metricbeat/module/googlecloud"
_ "github.com/elastic/beats/v7/x-pack/metricbeat/module/googlecloud/stackdriver"
_ "github.com/elastic/beats/v7/x-pack/metricbeat/module/googlecloud/metrics"
)

func init() {
Expand Down
2 changes: 1 addition & 1 deletion x-pack/metricbeat/module/googlecloud/compute/manifest.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
default: true
input:
module: googlecloud
metricset: stackdriver
metricset: metrics
defaults:
metrics:
- service: compute
Expand Down
2 changes: 1 addition & 1 deletion x-pack/metricbeat/module/googlecloud/fields.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import (

"github.com/elastic/beats/v7/libbeat/common"
mbtest "github.com/elastic/beats/v7/metricbeat/mb/testing"
"github.com/elastic/beats/v7/x-pack/metricbeat/module/googlecloud/stackdriver"
"github.com/elastic/beats/v7/x-pack/metricbeat/module/googlecloud/metrics"
)

func TestData(t *testing.T) {
Expand All @@ -34,7 +34,7 @@ func TestData(t *testing.T) {
{"googlecloud.loadbalancing.tcp_ssl_proxy", "./_meta/data_tcp_ssl_proxy.json"},
}

config := stackdriver.GetConfigForTest(t, "loadbalancing")
config := metrics.GetConfigForTest(t, "loadbalancing")

for _, df := range dataFiles {
metricSet := mbtest.NewFetcher(t, config)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import (

// Register input module and metricset
_ "github.com/elastic/beats/v7/x-pack/metricbeat/module/googlecloud"
_ "github.com/elastic/beats/v7/x-pack/metricbeat/module/googlecloud/stackdriver"
_ "github.com/elastic/beats/v7/x-pack/metricbeat/module/googlecloud/metrics"
)

func init() {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
default: false
input:
module: googlecloud
metricset: stackdriver
metricset: metrics
defaults:
metrics:
- service: loadbalancing
Expand Down
36 changes: 36 additions & 0 deletions x-pack/metricbeat/module/googlecloud/metrics/_meta/data.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"@timestamp": "2017-10-12T08:05:34.853Z",
"cloud": {
"account": {
"id": "elastic-apm"
},
"instance": {
"id": "8867356942891823986",
"name": "apm-ui-dev"
},
"provider": "googlecloud"
},
"cloud.availability_zone": "europe-west4-a",
"event": {
"dataset": "googlecloud.metrics",
"duration": 115000,
"module": "googlecloud"
},
"googlecloud": {
"labels": {},
"metrics": {
"instance": {
"uptime": {
"value": 60
}
}
}
},
"metricset": {
"name": "metrics",
"period": 10000
},
"service": {
"type": "googlecloud"
}
}
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
Stackdriver provides visibility into the performance, uptime, and overall health
Operations monitoring provides visibility into the performance, uptime, and overall health
of cloud-powered applications. It collects metrics, events, and metadata from
different services from Google Cloud. This metricset is to collect monitoring
metrics from Google Cloud using `ListTimeSeries` API. The full list of metric
types that Google Cloud monitoring supports can be found in
https://cloud.google.com/monitoring/api/metrics_gcp#gcp[Google Cloud Metrics].

Each monitoring metric from stackdriver has a sample period and/or ingest delay.
Each monitoring metric from Google Cloud has a sample period and/or ingest delay.
Sample period is the time interval between consecutive data points for metrics
that are written periodically. Ingest delay represents the time for data points
older than this value are guaranteed to be available to read. Sample period and
Expand All @@ -32,7 +32,7 @@ for the full list of aligners.

[float]
=== Example Configuration
* `stackdriver` metricset is enabled to collect metrics from all zones under
* `metrics` metricset is enabled to collect metrics from all zones under
`europe-west1-c` region in `elastic-observability` project. Two sets of metrics
are specified: first one is to collect CPU usage time and utilization with
aggregation aligner ALIGN_MEAN; second one is to collect uptime with aggregation
Expand All @@ -45,7 +45,7 @@ given aggregation aligner applied for each metric type.
----
- module: googlecloud
metricsets:
- stackdriver
- metrics
zone: "europe-west1-c"
project_id: elastic-observability
credentials_file_path: "your JSON credentials file path"
Expand All @@ -64,7 +64,7 @@ given aggregation aligner applied for each metric type.

----

* `stackdriver` metricset is enabled to collect metrics from all zones under
* `metrics` metricset is enabled to collect metrics from all zones under
`europe-west1-c` region in `elastic-observability` project. Two sets of metrics
are specified: first one is to collect CPU usage time and utilization with
aggregation aligner ALIGN_MEAN; second one is to collect uptime with aggregation
Expand All @@ -78,7 +78,7 @@ ignored.
----
- module: googlecloud
metricsets:
- stackdriver
- metrics
zone: "europe-west1-c"
project_id: elastic-observability
credentials_file_path: "your JSON credentials file path"
Expand All @@ -96,7 +96,7 @@ ignored.
- "instance/uptime"
----

* `stackdriver` metricset is enabled to collect metrics from all zones under
* `metrics` metricset is enabled to collect metrics from all zones under
`europe-west1-c` region in `elastic-observability` project. One set of metrics
will be collected: metric types that starts with `instance/cpu` under `compute`
service with aligner ALIGN_NONE. These metric types all have 240 seconds ingest
Expand All @@ -112,7 +112,7 @@ every minute with no aggregation. The metric types in `compute` service with
----
- module: googlecloud
metricsets:
- stackdriver
- metrics
zone: "europe-west1-c"
project_id: elastic-observability
credentials_file_path: "your JSON credentials file path"
Expand Down
Loading