Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AWS] add latency documentation #6231

Merged
merged 11 commits into from
Jun 12, 2023
21 changes: 18 additions & 3 deletions packages/aws/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ This may include every user request that CloudFront receives, every action taken
by an AWS user or role, and more.

**Metrics** give you insight into the state of your AWS services.
his may include understanding where you're spending the most and why, the volume of storage you're using,
This may include understanding where you're spending the most and why, the volume of storage you're using,
CPU utilization of your instances, and more.

For a complete list of all AWS services and the data streams available for each, see [Reference](#reference).
Expand Down Expand Up @@ -176,12 +176,27 @@ Use the AWS integration to connect to your AWS account and collect data from mul
When you configure the integration, you can collect data from as many AWS services as you'd like.

If you only need to collect data from one AWS service, consider using the individual integration
(for example, to only collect billing metrics, you can use the
**AWS CloudFront** integration).
(for example, to only collect monitoring metrics for EC2, you can configure only the
**AWS EC2** integration).

For step-by-step instructions on how to set up an integration, see the
[Getting started](https://www.elastic.co/guide/en/welcome-to-elastic/current/getting-started-observability.html) guide.

## Debug
### Latency causes missing metrics
Some AWS services send monitoring metrics to CloudWatch with a latency to process larger than the integration collection
period. This will cause data points missing or none get collected by the agent. In this case, please specify a
latency parameter so collection start time and end time will be shifted by the given latency amount.

In order to check how much the latency is, you can log into the AWS CloudWatch portal, find the latest data point that
showed up in AWS CloudWatch. Compare the timestamp of this latest data point with the current timestamp to see what's
the difference. This difference can be used as latency.

For example, the screenshot below is taken at `2023-05-09 22:30 UTC` and the timestamp for the last data point is
`2023-05-09 22:15 UTC`. This means there is a 15min delay between the current time and CloudWatch. With this information,
we should add a `latency` configuration for `15m` when adding the integration.
![alt text](../img/metricbeat-aws-cloudwatch-latency.png "CloudWatch Last Data Point Timestamp")

kaiyan-sheng marked this conversation as resolved.
Show resolved Hide resolved
## Reference

Below is an overview of the type of data you can collect from each AWS service.
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.