-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Awscloudwatchtags #41388
Merged
Merged
Awscloudwatchtags #41388
Changes from 57 commits
Commits
Show all changes
61 commits
Select commit
Hold shift + click to select a range
4eda24f
adding fix for aws tags of cloudwatch
gizas 1750c54
working for the first aws fix
gizas cf7db5d
first working sample for aws tags with filters
gizas 64de6bf
adding CHANGELOG.next
gizas 4bcb8df
Merge branch 'main' of github.com:elastic/beats into awscloudwatchtags
gizas 7db4e0e
adding CHANGELOG.next
gizas f4fd00f
adding CHANGELOG.next
gizas dc5e38b
minor changes
gizas 1479e8b
adding notice
gizas 2f3e2e0
Merge branch 'main' of github.com:elastic/beats into awscloudwatchtags
gizas 7aa4701
fixing also request for http gw
gizas 8ff8425
merging the http apigw
gizas 5369a07
added checks for collecting rest apis only in case the check fails
gizas 2750f39
added checks for collecting rest apis only in case the check fails
gizas b9ad7e6
added checks for collecting rest apis only in case the check fails
gizas 4f3d5d0
Merge branch 'main' into awscloudwatchtags
gizas ab38936
merging with main
gizas 98918ef
Update x-pack/metricbeat/module/aws/utils.go
gizas 63c29fb
Merge branch 'main' into awscloudwatchtags
gizas 2128d2b
merging with main
gizas 5ef7bd1
merging with main
gizas d0b2f07
lint error
gizas 94869b5
adding const and creating a struct to return values
gizas 4790a81
Merge branch 'main' into awscloudwatchtags
gizas e51eac0
fixing conflicts
gizas d2117fe
Merge branch 'awscloudwatchtags' of github.com:elastic/beats into aws…
gizas 7bd7324
adding vars from const
gizas 38c8cae
adding vars from const
gizas ac27bbe
Merge branch 'awscloudwatchtags' of github.com:elastic/beats into aws…
gizas 71250b6
Merge branch 'main' into awscloudwatchtags
gizas 0acaf51
updating docs
gizas 58cd991
Merge branch 'main' into awscloudwatchtags
gizas 211a321
Update x-pack/metricbeat/module/aws/cloudwatch/cloudwatch_test.go
gizas 03bd34a
Update x-pack/metricbeat/module/aws/cloudwatch/cloudwatch.go
gizas 535b905
moving vars up and merging with main
gizas 26a4375
adding a validation for the LimitRestAPI
gizas 704314d
Merge branch 'main' into awscloudwatchtags
gizas 860cc0d
Merge branch 'main' into awscloudwatchtags
gizas 5e2c6a9
Merge branch 'main' into awscloudwatchtags
gizas 705ffb5
merging with main
gizas c1e2cf2
merging with main
gizas 17cd9f4
Merge branch 'main' into awscloudwatchtags
gizas 1ce7cbc
Merge branch 'main' into awscloudwatchtags
gizas edd5a8f
fixing conflixts with go.sum
gizas a753220
Merge branch 'awscloudwatchtags' of github.com:elastic/beats into aws…
gizas d1894cd
fixing docs
gizas dce6fd2
fixing docs
gizas a4788ea
fixing docs
gizas d65b409
variables set to apigateway_max_results
gizas c5139b3
moving check inside apigw namespace
gizas f95c8e4
setting to max limit to 500
gizas e2376ea
setting to max limit to 500
gizas c7d7980
merging conflicts with main
gizas 03c9019
Merge branch 'main' into awscloudwatchtags
gizas 10c6f02
Merge branch 'main' into awscloudwatchtags
gizas 687c010
Merge branch 'main' into awscloudwatchtags
gizas eee6436
Merge branch 'main' into awscloudwatchtags
gizas 84868c0
removing uneeded sdk
gizas 262916e
Merge branch 'awscloudwatchtags' of github.com:elastic/beats into aws…
gizas 2844563
adding Notice.txt
gizas 6a84843
Merge branch 'main' into awscloudwatchtags
gizas File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -146,6 +146,25 @@ Enforces the use of FIPS service endpoints. See <<aws-credentials-config,AWS cre | |
- ec2 | ||
---- | ||
|
||
* *apigateway_max_results* | ||
|
||
This configuration works together with AWS/APIGateway namespace. It defines the maximum number of returned results per page. The default value is 25 and the maximum value is 500. | ||
See https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/[email protected]#GetRestApisInput.Limit[GetRestApisInput.Limit] | ||
|
||
[source,yaml] | ||
---- | ||
- module: aws | ||
period: 10s | ||
regions: | ||
- us-east-1 | ||
metricsets: | ||
- cloudwatch | ||
metrics: | ||
- namespace: "AWS/ApiGateway" | ||
resource_type: "apigateway:restapis" | ||
apigateway_max_results: 40 | ||
---- | ||
|
||
The aws module comes with a predefined dashboard. For example: | ||
|
||
image::./images/metricbeat-aws-overview.png[] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -134,6 +134,25 @@ Enforces the use of FIPS service endpoints. See <<aws-credentials-config,AWS cre | |
- ec2 | ||
---- | ||
|
||
* *apigateway_max_results* | ||
|
||
This configuration works together with AWS/APIGateway namespace. It defines the maximum number of returned results per page. The default value is 25 and the maximum value is 500. | ||
See https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/[email protected]#GetRestApisInput.Limit[GetRestApisInput.Limit] | ||
|
||
[source,yaml] | ||
---- | ||
- module: aws | ||
period: 10s | ||
regions: | ||
- us-east-1 | ||
metricsets: | ||
- cloudwatch | ||
metrics: | ||
- namespace: "AWS/ApiGateway" | ||
resource_type: "apigateway:restapis" | ||
apigateway_max_results: 40 | ||
---- | ||
|
||
The aws module comes with a predefined dashboard. For example: | ||
|
||
image::./images/metricbeat-aws-overview.png[] | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is deprecated and goes EOL in July 2025, when you'll be forced to remove it because it won't get security updates (or I guess you could wait until there is a CVE reported in it and then urgently remove it): https://github.com/aws/aws-sdk-go
This PR also makes agentbeat 5% larger. Do we really need both versions of the AWS SDK?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The whole idea of this fix was that we needed to query both Rest APIs (that were included in old sdk) and WebSocket and HTTP APIs (that are included in the v2 sdk) in case user needed the metadata
(See code for check)
Let me doublecheck again and let you know. I am not still merging this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@cmacknz I was not accurate, the
https://github.com/aws/aws-sdk-go
was not needed. Now it is removed!Long explanation:
I was wrongly using aws.StringValue (See old commit) from old sdk, in order to be safe with nill pointer exception
I found a replacer with aws.Tostring from new sdk (see new code)
Just to clarify for the following code, both functions use
aws-sdk-go-v2
but: