address review comments

elastic · Nov 21, 2024 · 50d9adc · 50d9adc
1 parent 3e04de8
commit 50d9adc
Show file tree

Hide file tree

Showing 14 changed files with 663 additions and 647 deletions.
diff --git a/packages/vsphere/_dev/build/docs/README.md b/packages/vsphere/_dev/build/docs/README.md
@@ -16,149 +16,143 @@ The vSphere integration collects logs and metrics.
 
 Logs help you keep a record of events that happen on your machine. The `Log` data stream collected by vSphere as integration is `log`.
 
-Metrics give you insight into the statistics of the vSphere. The `Metric` data streams collected by the vSphere integration are `cluster`, `datastore`, `datastorecluster`, `host`, `network`, `resourcepool` and `virtualmachine` so that the user can monitor and troubleshoot the performance of the vSphere instance.
+Metrics give you insight into the statistics of the vSphere. The `Metric` data stream collected by the vSphere integration are `cluster`, `datastore`, `datastorecluster`, `host`, `network`, `resourcepool` and `virtualmachine` so that the user can monitor and troubleshoot the performance of the vSphere instance.
+
+Data Streams:
+
+- **`log`**: This data stream collects logs generated by VMware vSphere using a syslog daemon.
+- **`cluster`**: This data stream collects metrics from VMware vSphere, such as lists of datastores, hosts and networks associated with cluster.
+- **`datastore`**: This data stream gathers datastore metrics from VMware vSphere, including performance statistics such as capacity, usage, read/write operations, latency, and throughput.
+- **`datastorecluster`**: This data stream gathers metrics for datastore clusters from VMware vSphere, including statistics like cluster capacity and available free space. Additionally, it provides information about the individual datastores that comprise the cluster.
+- **`host`**: This data stream collects host metrics from VMware vSphere, including performance statistics such as CPU usage, memory usage, disk I/O, and network activity.
+- **`resourcepool`**: This data stream collects metrics from VMware vSphere, such as CPU and memory usage, CPU and memory reservation, and CPU and memory limit.
+- **`virtualmachine`**: This data stream gathers virtual machine metrics from VMware vSphere, including performance statistics such as status, uptime, CPU usage, memory usage, and network activity.
+- **`network`**: This data stream gathers metrics and status information related to VMware vSphere networks, including network accessibility, connected hosts and virtual machines, configuration health, and network type.
+
+## Important Note
+- Users can monitor and see the log inside the ingested documents for vSphere in the `logs-*` index pattern from `Discover`, and for metrics, the index pattern is `metrics-*`.
+- **Supported Periods**:
+    - Real-time data collection – An ESXi Server collects data for each performance counter every 20 seconds by default.
+    - The Datastore and Host datastreams support performance data collection using the vSphere performance API.
+    - Since the performance API has usage restrictions based on data collection intervals, users should ensure that the period is configured optimally to receive real-time data. Users can still collect summary metrics if performance metrics are not supported for the configured instance.
+    - Period configuration can be determined based on the [Data Collection Intervals](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.monitoring.doc/GUID-247646EA-A04B-411A-8DD4-62A3DCFCF49B.html) and [Data Collection Levels](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.monitoring.doc/GUID-25800DE4-68E5-41CC-82D9-8811E27924BC.html).
 
-Data streams:
-
-- **`log`**: Collects logs generated by VMware vSphere using a syslog daemon.
-- **`cluster`**: Collects metrics from VMware vSphere, such as lists of datastores, hosts and networks associated with cluster.
-- **`datastore`**: Gathers datastore metrics from VMware vSphere, including performance statistics such as capacity, usage, read/write operations, latency, and throughput.
-- **`datastorecluster`**: Gathers metrics for datastore clusters from VMware vSphere, including statistics like cluster capacity and available free space. Additionally, it provides information about the individual datastores that comprise the cluster.
-- **`host`**: Collects host metrics from VMware vSphere, including performance statistics such as CPU usage, memory usage, disk I/O, and network activity.
-- **`resourcepool`**: Collects metrics from VMware vSphere, such as CPU and memory usage, CPU and memory reservation, and CPU and memory limit.
-- **`virtualmachine`**: Gathers virtual machine metrics from VMware vSphere, including performance statistics such as status, uptime, CPU usage, memory usage, and network activity.
-- **`network`**: Gathers metrics and status information related to VMware vSphere networks, including network accessibility, connected hosts and virtual machines, configuration health, and network type.
-
-Note:
-
-Users can monitor and check the log inside the ingested documents for vSphere in the `logs-*` index pattern from `Discover`, and for metrics, the index pattern is `metrics-*`.
+## Prerequisites
 
-## Supported periods
-
-- Real-time data collection: An ESXi Server collects data for each performance counter every 20 seconds by default.
-- The `datastore` and `host` data streams support performance data collection using the vSphere performance API.
-- Since the performance API has usage restrictions based on data collection intervals, make sure that the period is configured optimally to receive real-time data. You can still collect summary metrics if performance metrics are not supported for the configured instance.
-- Period configuration can be determined based on the [Data Collection Intervals](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.monitoring.doc/GUID-247646EA-A04B-411A-8DD4-62A3DCFCF49B.html) and [Data Collection Levels](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.monitoring.doc/GUID-25800DE4-68E5-41CC-82D9-8811E27924BC.html).
+You can store and search your data using Elasticsearch and visualize and manage it with Kibana. We recommend using our hosted Elasticsearch Service on Elastic Cloud or self-managing the Elastic Stack on your own hardware.
 
 ## Compatibility
 
 This integration has been tested and verified against VMware ESXi and vCenter version 7.0.3.
 
-## Prerequisites
-
-You can store and search your data using Elasticsearch and visualize and manage it with Kibana. We recommend using our hosted Elasticsearch Service on Elastic Cloud or self-managing the Elastic Stack on your own hardware.
-
 ## Setup
 
-For step-by-step instructions on how to set up an integration, check the [Getting started](https://www.elastic.co/guide/en/welcome-to-elastic/current/getting-started-observability.html) guide.
+For step-by-step instructions on how to set up an integration, see the [Getting started](https://www.elastic.co/guide/en/welcome-to-elastic/current/getting-started-observability.html) guide.
 
-## Logs
+## Logs reference
 
-The `log` data stream collects logs generated by VMware vSphere using a syslog daemon.
+### Log
 
-To collect logs, follow these steps:
+This is the `log` data stream. This data stream collects logs generated by VMware vSphere using a syslog daemon.
 
-1. Configure the listening host/IP address (default: localhost) and host port (default: 9525) in the integration. 
-
-1. Configure vSphere to send logs to a remote syslog host and provide the configured hostname/IP and port of the Elastic Agent host.
+Note:
+- To collect logs, a syslog daemon is used. First, you must configure the listening host/IP address (default: localhost) and host port (default: 9525) in the integration. Then, configure vSphere to send logs to a remote syslog host and provide the configured hostname/IP and port of the Elastic Agent host.
 
 {{event "log"}}
 
 **ECS Field Reference**
 
-Refer to the [ECS Field Reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
+Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
 
 {{fields "log"}}
 
-## Metrics
+## Metrics reference
 
-To access the metrics, provide the URL <https://host:port/sdk> in the **Add Integration** page of the vSphere package.
+Note:
+- To access the metrics, provide the URL <https://host:port/sdk> in the "Add Integration" page of the vSphere package.
 
 ### Cluster
-
 Clusters in vSphere represent a group of ESXi hosts working together to optimize resource allocation, ensure high availability, and manage workloads efficiently.
 
 {{event "cluster"}}
 
 **ECS Field Reference**
 
-Refer to the [ECS Field Reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
+Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
 
 {{fields "cluster"}}
 
 ### Datastore
 
-This data stream collects datastore metrics from VMware vSphere, including performance statistics such as capacity, usage, read/write operations, latency, and throughput.
+This is `datastore` data stream. This data stream collects datastore metrics from VMware vSphere, including performance statistics such as capacity, usage, read/write operations, latency, and throughput.
 
 {{event "datastore"}}
 
 **ECS Field Reference**
 
-Refer to the [ECS Field Reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
+Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
 
 {{fields "datastore"}}
 
-### Datastore cluster
+### Datastore Cluster
 
 Datastore clusters in vSphere group multiple datastores for optimized management and automated load balancing, ensuring efficient storage utilization and simplified administration.
 
 {{event "datastorecluster"}}
 
 **ECS Field Reference**
 
-Refer to the [ECS Field Reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
+Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
 
 {{fields "datastorecluster"}}
 
 ### Host
 
-This data stream collects host metrics from VMware vSphere, including performance statistics such as CPU usage, memory usage, disk I/O, and network activity.
+This is the `host` data stream. This data stream collects host metrics from VMware vSphere, including performance statistics such as CPU usage, memory usage, disk I/O, and network activity.
 
 {{event "host"}}
 
 **ECS Field Reference**
 
-Refer to the [ECS Field Reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
+Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
 
 {{fields "host"}}
 
 ### Network
 
-This data stream gathers metrics and status information related to VMware vSphere networks, including network accessibility, connected hosts and virtual machines, configuration health, and network type.
+This is the `network` data stream. This data stream gathers metrics and status information related to VMware vSphere networks, including network accessibility, connected hosts and virtual machines, configuration health, and network type.
 
 {{event "network"}}
 
 **ECS Field Reference**
 
-Refer to the [ECS Field Reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
+Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
 
 {{fields "network"}}
 
 ### Resourcepool
-
-Resource pools in vSphere enable the allocation and management of CPU and memory resources across groups of virtual machines.
+Resource pools in vSphere allow for the allocation and management of CPU and memory resources across groups of virtual machines.
 
 {{event "resourcepool"}}
 
 **ECS Field Reference**
 
-Refer to the [ECS Field Reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
+Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
 
 {{fields "resourcepool"}}
 
 ### Virtual Machine
 
-This data stream collects virtual machine metrics from VMware vSphere, including performance statistics such as status, uptime, CPU usage, memory usage, and network activity.
+This is the `virtualmachine` data stream. This data stream collects virtual machine metrics from VMware vSphere, including performance statistics such as status, uptime, CPU usage, memory usage, and network activity.
 
 {{event "virtualmachine"}}
 
 **ECS Field Reference**
 
-Refer to the [ECS Field Reference](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
+Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
 
 {{fields "virtualmachine"}}
 
 ## Troubleshoot
 
-If you encounter issues during the setup or usage of the VMware vSphere integration, in particular with Data Collection Intervals, with agent error messages such as **`Failed to query performance metrics: ServerFaultCode: A specified parameter was not correct: querySpec.interval`**, refer to 
-[Supported periods](#supported-periods) for guidance and resolution.
+If you encounter any issues during the setup or usage of the VMware vSphere integration, particularly with regards to Data Collection Intervals, with agent error messages such as **`Failed to query performance metrics: ServerFaultCode: A specified parameter was not correct: querySpec.interval`**, please refer to the mentioned Important Notes/Supported Periods for guidance and resolution.
diff --git a/packages/vsphere/data_stream/cluster/fields/fields.yml b/packages/vsphere/data_stream/cluster/fields/fields.yml
@@ -5,7 +5,7 @@
     - name: alert.names
       type: keyword
       description: >
-        List of all the alerts on this Cluster.
+        List of all the alerts on this cluster.
     - name: das_config
       type: group
       fields:
@@ -91,4 +91,4 @@
     - name: warning.names
       type: keyword
       description: >
-        List of all the warnings on this Cluster.
+        List of all the warnings on this cluster.
diff --git a/packages/vsphere/data_stream/cluster/sample_event.json b/packages/vsphere/data_stream/cluster/sample_event.json
@@ -1,54 +1,51 @@
 {
-    "@timestamp": "2024-09-19T05:44:00.800Z",
+    "@timestamp": "2024-11-21T06:39:19.834Z",
     "agent": {
-        "ephemeral_id": "676a770b-a207-4fec-99d4-e82377578711",
-        "id": "6b430ae3-0bdb-4d5c-b60d-a02f54e770e5",
-        "name": "elastic-agent-47605",
+        "ephemeral_id": "f41470c3-9bf0-4217-baf6-73383f35fcff",
+        "id": "6b90cdc8-92f4-486f-9b8a-02a0aca316a6",
+        "name": "elastic-agent-81320",
         "type": "metricbeat",
-        "version": "8.15.2"
+        "version": "8.17.0"
     },
     "data_stream": {
         "dataset": "vsphere.cluster",
-        "namespace": "93141",
+        "namespace": "35689",
         "type": "metrics"
     },
     "ecs": {
         "version": "8.11.0"
     },
     "elastic_agent": {
-        "id": "6b430ae3-0bdb-4d5c-b60d-a02f54e770e5",
+        "id": "6b90cdc8-92f4-486f-9b8a-02a0aca316a6",
         "snapshot": true,
-        "version": "8.15.2"
+        "version": "8.17.0"
     },
     "event": {
         "agent_id_status": "verified",
         "dataset": "vsphere.cluster",
-        "duration": 17059144,
-        "ingested": "2024-09-19T05:44:03Z",
+        "duration": 33840300,
+        "ingested": "2024-11-21T06:39:22Z",
         "module": "vsphere"
     },
     "host": {
         "architecture": "x86_64",
         "containerized": true,
-        "hostname": "elastic-agent-47605",
-        "id": "57723763cd1b4ff48e54a505de4ebe6c",
+        "hostname": "elastic-agent-81320",
         "ip": [
-            "192.168.244.4",
-            "192.168.245.2"
+            "192.168.242.4",
+            "192.168.243.2"
         ],
         "mac": [
-            "02-42-C0-A8-F4-04",
-            "02-42-C0-A8-F5-02"
+            "02-42-C0-A8-F2-04",
+            "02-42-C0-A8-F3-02"
         ],
-        "name": "elastic-agent-47605",
+        "name": "elastic-agent-81320",
         "os": {
-            "codename": "focal",
-            "family": "debian",
-            "kernel": "4.18.0-348.7.1.el8_5.x86_64",
-            "name": "Ubuntu",
-            "platform": "ubuntu",
+            "kernel": "3.10.0-1160.118.1.el7.x86_64",
+            "name": "Wolfi",
+            "platform": "wolfi",
             "type": "linux",
-            "version": "20.04.6 LTS (Focal Fossa)"
+            "version": "20230201"
         }
     },
     "metricset": {
@@ -76,15 +73,16 @@
                     "DC0_C0_H2"
                 ]
             },
+            "id": "domain-c28",
             "name": "DC0_C0",
             "network": {
                 "count": 3,
                 "names": [
                     "DC0_DVPG0",
-                    "DVS0-DVUplinks-9",
+                    "DVS0-DVUplinks-10",
                     "VM Network"
                 ]
             }
         }
     }
-}
+}
diff --git a/packages/vsphere/data_stream/datastore/fields/fields.yml b/packages/vsphere/data_stream/datastore/fields/fields.yml
@@ -5,7 +5,7 @@
     - name: alert.names
       type: keyword
       description: >
-        List of all the alerts on this Datastore.
+        List of all the alerts on this datastore.
     - name: name
       type: keyword
       # Reason to add as a dimension field: to uniquely identify the datastore