-
Notifications
You must be signed in to change notification settings - Fork 4.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
Co-authored-by: DeDe Morton <[email protected]>
- Loading branch information
1 parent
5ea5313
commit 923e11f
Showing
7 changed files
with
37 additions
and
246 deletions.
There are no files selected for viewing
114 changes: 24 additions & 90 deletions
114
x-pack/metricbeat/module/gcp/compute/_meta/docs.asciidoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,96 +1,30 @@ | ||
| Compute metricset to fetch metrics from https://cloud.google.com/compute/[Compute Engine] Virtual Machines in Google Cloud Platform. No Monitoring or Logging agent is required in your instances to use this metricset. | ||
|
|
||
| The `compute` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-compute[Stackdriver API]. The field names have been left untouched for people already familiar with them. | ||
| The `compute` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-compute[Stackdriver API]. The field names are aligned to {beats-devguide}/event-conventions.html[Beats naming conventions] with minor modifications to their GCP metrics name counterpart. | ||
|
|
||
| Extra labels and metadata are also extracted using the https://cloud.google.com/compute/docs/reference/rest/v1/instances/get[Compute API]. This is enough to get most of the info associated with a metric like Compute labels and metadata and metric specific Labels. | ||
|
|
||
| [float] | ||
| === Metrics and labels | ||
| Here is a list of metrics collected by `compute` metricset: | ||
|
|
||
| [float] | ||
| ==== firewall | ||
|
|
||
| * `instance.firewall.dropped_bytes_count`: Incoming bytes dropped by the firewall. | ||
| - `instance_name`: The name of the VM instance. | ||
|
|
||
| * `instance.firewall.dropped_packets_count`: Incoming packets dropped by the firewall. | ||
| - `instance_name`: The name of the VM instance. | ||
|
|
||
| [float] | ||
| ==== cpu | ||
|
|
||
| * `instance.cpu.reserved_cores`: Number of cores reserved on the host of the `instance`. | ||
| - `instance_name`: The name of the VM instance. | ||
|
|
||
| * `instance.cpu.utilization`: The fraction of the allocated CPU that is currently in use on the `instance`. | ||
| - `instance_name`: The name of the VM instance. | ||
|
|
||
| * `instance.cpu.usage_time`: Usage for all cores in seconds. | ||
| - `instance_name`: The name of the VM instance. | ||
|
|
||
| [float] | ||
| ==== disk | ||
|
|
||
| * `instance.disk.read_bytes_count`: Count of bytes read from disk. | ||
| - `instance_name`: The name of the VM instance. | ||
| - `device_name`: The name of the disk device. | ||
| - `storage_type`: The storage type: `pd-standard`, `pd-ssd`, or `local-ssd`. | ||
| - `device_type`: The disk type: `ephemeral` or `permanent`. | ||
|
|
||
| * `instance.disk.read_ops_count`: Count of disk read IO operations. | ||
| - `instance_name`: The name of the VM instance. | ||
| - `device_name`: The name of the disk device. | ||
| - `storage_type`: The storage type: `pd-standard`, `pd-ssd`, or `local-ssd`. | ||
| - `device_type`: The disk type: `ephemeral` or `permanent`. | ||
|
|
||
| * `instance.disk.write_bytes_count`: Count of bytes written to disk | ||
| - `instance_name`: The name of the VM instance. | ||
| - `device_name`: The name of the disk device. | ||
| - `storage_type`: The storage type: `pd-standard`, `pd-ssd`, or `local-ssd`. | ||
| - `device_type`: The disk type: `ephemeral` or `permanent`. | ||
|
|
||
| * `instance.disk.write_ops_count`: Count of disk write IO operations. | ||
| - `instance_name`: The name of the VM instance. | ||
| - `device_name`: The name of the disk device. | ||
| - `storage_type`: The storage type: `pd-standard`, `pd-ssd`, or `local-ssd`. | ||
| - `device_type`: The disk type: `ephemeral` or `permanent`. | ||
|
|
||
| [float] | ||
| ==== uptime | ||
|
|
||
| * `instance.uptime`: How long the VM has been running, in seconds | ||
| - `instance_name`: The name of the VM instance. | ||
|
|
||
| [float] | ||
| ==== network | ||
|
|
||
| * `instance.network.received_bytes_count`: Count of bytes received from the network | ||
| - `instance_name`: The name of the VM instance. | ||
| - `loadBalanced`: Whether traffic was sent from an L3 loadbalanced IP address assigned to the VM. Traffic that is externally routed from the VM's standard internal or external IP address, such as L7 loadbalanced traffic, is not considered to be loadbalanced in this metric. | ||
|
|
||
| * `instance.network.received_packets_count`: Count of packets received from the network | ||
| - `instance_name`: The name of the VM instance. | ||
| - `loadBalanced`: Whether traffic was sent from an L3 loadbalanced IP address assigned to the VM. Traffic that is externally routed from the VM's standard internal or external IP address, such as L7 loadbalanced traffic, is not considered to be loadbalanced in this metric. | ||
|
|
||
| * `instance.network.sent_bytes_count`: Count of bytes sent over the network | ||
| - `instance_name`: The name of the VM instance. | ||
| - `loadBalanced`: Whether traffic was sent from an L3 loadbalanced IP address assigned to the VM. Traffic that is externally routed from the VM's standard internal or external IP address, such as L7 loadbalanced traffic, is not considered to be loadbalanced in this metric. | ||
|
|
||
| * `instance.network.sent_packets_count`: Count of packets sent over the network | ||
| - `instance_name`: The name of the VM instance. | ||
| - `loadBalanced`: Whether traffic was sent from an L3 loadbalanced IP address assigned to the VM. Traffic that is externally routed from the VM's standard internal or external IP address, such as L7 loadbalanced traffic, is not considered to be loadbalanced in this metric. | ||
|
|
||
| [float] | ||
| ==== memory | ||
| * `instance.memory.balloon.ram_size`: The total amount of memory in the VM. This metric is only available for VMs that belong to the e2 family. | ||
| - `instance_name`: The name of the VM instance. | ||
|
|
||
| * `instance.memory.balloon.ram_used`: Memory currently used in the VM. This metric is only available for VMs that belong to the e2 family. | ||
| - `instance_name`: The name of the VM instance. | ||
|
|
||
| * `instance.memory.balloon.swap_in_bytes_count`: The amount of memory read into the guest from its own swap space. This metric is only available for VMs that belong to the e2 family. | ||
| - `instance_name`: The name of the VM instance. | ||
|
|
||
| * `instance.memory.balloon.swap_out_bytes_count`: The amount of memory written from the guest to its own swap space. This metric is only available for VMs that belong to the e2 family. | ||
| - `instance_name`: The name of the VM instance. | ||
| === Labels | ||
| Here is a list of labels collected by `compute` metricset depending on the type of metric being collected: | ||
|
|
||
| * `instance_name`: The name of the VM instance. | ||
| Collected with: | ||
| ** `gcp.instance.firewall.*` | ||
| ** `gcp.instance.cpu.*` | ||
| ** `gcp.instance.disk.*` | ||
| ** `gcp.instance.memory.*` | ||
| ** `gcp.instance.network.*` | ||
| ** `gcp.instance.uptime` | ||
| * `device_name`: The name of the disk device. | ||
| Collected with: | ||
| ** `gcp.instance.disk.*` | ||
| * `storage_type`: The storage type: `pd-standard`, `pd-ssd`, or `local-ssd`. | ||
| Collected with: | ||
| ** `gcp.instance.disk.*` | ||
| * `device_type`: The disk type: `ephemeral` or `permanent`. | ||
| Collected with: | ||
| ** `gcp.instance.disk.*` | ||
| * `loadBalanced`: Whether traffic was sent from an L3 loadbalanced IP address assigned to the VM. Traffic that is externally routed from the VM's standard internal or external IP address, such as L7 loadbalanced traffic, is not considered to be loadbalanced in this metric. | ||
| Collected with: | ||
| ** `gcp.instance.network.*` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,32 +1,5 @@ | ||
| Dataproc metricset fetches metrics from https://cloud.google.com/dataproc/[Dataproc] in Google Cloud Platform. | ||
|
|
||
| The `dataproc` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-dataproc[GCP Dataproc Monitoring API]. The field names have been left untouched for people already familiar with them. | ||
| The `dataproc` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-dataproc[GCP Dataproc Monitoring API]. The field names are aligned to {beats-devguide}/event-conventions.html[Beats naming conventions] with minor modifications to their GCP metrics name counterpart. | ||
|
|
||
| You can specify a single region to fetch metrics like `us-central1`. Be aware that GCP Storage does not use zones so `us-central1-a` will return nothing. If no region is specified, it will return metrics from all buckets. | ||
|
|
||
| [float] | ||
| === Metrics | ||
| Here is a list of metrics collected by `dataproc` metricset: | ||
|
|
||
| - `dataproc.batch.spark.executors`: Indicates the number of Batch Spark executors. | ||
| - `dataproc.cluster.hdfs.datanodes`: Indicates the number of HDFS DataNodes that are running inside a cluster. | ||
| - `dataproc.cluster.hdfs.storage_capacity`: Indicates capacity of HDFS system running on cluster in GB. | ||
| - `dataproc.cluster.hdfs.storage_utilization`: The percentage of HDFS storage currently used. | ||
| - `dataproc.cluster.hdfs.unhealthy_blocks`: Indicates the number of unhealthy blocks inside the cluster. | ||
| - `dataproc.cluster.job.completion_time`: The time jobs took to complete from the time the user submits a job to the time Dataproc reports it is completed. | ||
| - `dataproc.cluster.job.duration`: The time jobs have spent in a given state. | ||
| - `dataproc.cluster.job.failed_count`: Indicates the number of jobs that have failed on a cluster. | ||
| - `dataproc.cluster.job.running_count`: Indicates the number of jobs that are running on a cluster. | ||
| - `dataproc.cluster.job.submitted_count`: Indicates the number of jobs that have been submitted to a cluster. | ||
| - `dataproc.cluster.operation.completion_time`: The time operations took to complete from the time the user submits a operation to the time Dataproc reports it is completed. | ||
| - `dataproc.cluster.operation.duration`: The time operations have spent in a given state. | ||
| - `dataproc.cluster.operation.failed_count`: Indicates the number of operations that have failed on a cluster. | ||
| - `dataproc.cluster.operation.running_count`: Indicates the number of operations that are running on a cluster. | ||
| - `dataproc.cluster.operation.submitted_count`: Indicates the number of operations that have been submitted to a cluster. | ||
| - `dataproc.cluster.yarn.allocated_memory_percentage`: The percentage of YARN memory is allocated. | ||
| - `dataproc.cluster.yarn.apps`: Indicates the number of active YARN applications. | ||
| - `dataproc.cluster.yarn.containers`: Indicates the number of YARN containers. | ||
| - `dataproc.cluster.yarn.memory_size`: Indicates the YARN memory size in GB. Sampled every 60 seconds. | ||
| - `dataproc.cluster.yarn.nodemanagers`: Indicates the number of YARN NodeManagers running inside cluster. | ||
| - `dataproc.cluster.yarn.pending_memory_size`: The current memory request, in GB, that is pending to be fulfilled by the scheduler. | ||
| - `dataproc.cluster.yarn.virtual_cores`: Indicates the number of virtual cores in YARN. | ||
| You can specify a single region to fetch metrics like `us-central1`. Be aware that GCP Storage does not use zones so `us-central1-a` will return nothing. If no region is specified, metrics are returned from all buckets. |
12 changes: 2 additions & 10 deletions
12
x-pack/metricbeat/module/gcp/firestore/_meta/docs.asciidoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,5 @@ | ||
| Firestore metricset fetches metrics from https://cloud.google.com/firestore/[Firestore] in Google Cloud Platform. | ||
|
|
||
| The `firestore` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-firestore[GCP Firestore Monitoring API]. The field names have been left untouched for people already familiar with them. | ||
| The `firestore` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-firestore[GCP Firestore Monitoring API]. The field names are aligned to {beats-devguide}/event-conventions.html[Beats naming conventions] with minor modifications to their GCP metrics name counterpart. | ||
|
|
||
| You can specify a single region to fetch metrics like `us-central1`. Be aware that GCP Storage does not use zones so `us-central1-a` will return nothing. If no region is specified, it will return metrics from all buckets. | ||
|
|
||
| [float] | ||
| === Metrics | ||
| Here is a list of metrics collected by `firestore` metricset: | ||
|
|
||
| - `firestore.document.delete_count`: The number of successful document deletes. | ||
| - `firestore.document.read_count`: The number of successful document reads from queries or lookups. | ||
| - `firestore.document.write_count`: The number of successful document writes. | ||
| You can specify a single region to fetch metrics like `us-central1`. Be aware that GCP Storage does not use zones so `us-central1-a` will return nothing. If no region is specified, metrics are returned from all buckets. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
33 changes: 1 addition & 32 deletions
33
x-pack/metricbeat/module/gcp/loadbalancing/_meta/docs.asciidoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,34 +1,3 @@ | ||
| Load Balancing metricset fetches metrics from https://cloud.google.com/load-balancing/[Load Balancing] in Google Cloud Platform. | ||
|
|
||
| The `loadbalancing` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-loadbalancing[Stackdriver API]. The field names have been left untouched for people already familiar with them. | ||
|
|
||
| [float] | ||
| === Metrics | ||
| Here is a list of metrics collected by `loadbalancing` metricset: | ||
|
|
||
| [float] | ||
| ==== https | ||
|
|
||
| - `loadbalancing.https.backend_request_bytes_count`: The number of bytes sent as requests from HTTP/S load balancer to backends. | ||
| - `loadbalancing.https.backend_request_count`: The number of requests served by backends of HTTP/S load balancer. | ||
| - `loadbalancing.https.backend_response_bytes_count`: The number of bytes sent as responses from backends (or cache) to HTTP/S load balancer. | ||
| - `loadbalancing.https.request_bytes_count`: The number of bytes sent as requests from clients to HTTP/S load balancer. | ||
| - `loadbalancing.https.request_count`: The number of requests served by HTTP/S load balancer. | ||
| - `loadbalancing.https.response_bytes_count`: The number of bytes sent as responses from HTTP/S load balancer to clients. | ||
|
|
||
| [float] | ||
| ==== l3 | ||
|
|
||
| - `loadbalancing.l3.internal.egress_bytes_count`: The number of bytes sent from ILB backend to client (for TCP flows it's counting bytes on application stream only). | ||
| - `loadbalancing.l3.internal.egress_packets_count`: The number of packets sent from ILB backend to client of the flow. | ||
| - `loadbalancing.l3.internal.ingress_bytes_count`: The number of bytes sent from client to ILB backend (for TCP flows it's counting bytes on application stream only). | ||
| - `loadbalancing.l3.internal.ingress_packets_count`: The number of packets sent from client to ILB backend. | ||
|
|
||
| [float] | ||
| ==== tcp_ssl_proxy | ||
|
|
||
| - `loadbalancing.tcp_ssl_proxy.closed_connections`: Number of connections that were terminated over TCP/SSL proxy. | ||
| - `loadbalancing.tcp_ssl_proxy.egress_bytes_count`: Number of bytes sent from VM to client using proxy. | ||
| - `loadbalancing.tcp_ssl_proxy.ingress_bytes_count`: Number of bytes sent from client to VM using proxy. | ||
| - `loadbalancing.tcp_ssl_proxy.new_connections`: Number of connections that were created over TCP/SSL proxy. | ||
| - `loadbalancing.tcp_ssl_proxy.open_connections`: Current number of outstanding connections through the TCP/SSL proxy. | ||
| The `loadbalancing` metricset contains all metrics exported from the https://cloud.google.com/monitoring/api/metrics_gcp#gcp-loadbalancing[Stackdriver API]. The field names are aligned to {beats-devguide}/event-conventions.html[Beats naming conventions] with minor modifications to their GCP metrics name counterpart. |
Oops, something went wrong.