Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds findings in bucket level monitor #636

Merged
merged 3 commits into from
Nov 5, 2022
Merged

Conversation

eirsep
Copy link
Member

@eirsep eirsep commented Nov 3, 2022

This change surfaces out the list of documents that become a part of bucket level monitor aggregation buckets as findings for the monitor.

Signed-off-by: Surya Sashank Nistala [email protected]

CheckList:
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@eirsep eirsep requested a review from a team November 3, 2022 17:36
@eirsep eirsep requested a review from sbcd90 November 3, 2022 17:50
Signed-off-by: Surya Sashank Nistala <[email protected]>
@codecov-commenter
Copy link

codecov-commenter commented Nov 4, 2022

Codecov Report

Merging #636 (6b63fb7) into main (3491fb2) will decrease coverage by 0.22%.
The diff coverage is 83.90%.

@@             Coverage Diff              @@
##               main     #636      +/-   ##
============================================
- Coverage     76.35%   76.13%   -0.23%     
  Complexity      116      116              
============================================
  Files           124      124              
  Lines          6445     6569     +124     
  Branches        942      972      +30     
============================================
+ Hits           4921     5001      +80     
- Misses         1044     1070      +26     
- Partials        480      498      +18     
Impacted Files Coverage Δ
...rg/opensearch/alerting/BucketLevelMonitorRunner.kt 79.62% <83.33%> (+2.18%) ⬆️
...ain/kotlin/org/opensearch/alerting/AlertService.kt 79.06% <100.00%> (+0.09%) ⬆️
.../alerting/transport/TransportIndexMonitorAction.kt 64.65% <0.00%> (-5.56%) ⬇️
...destinationmigration/DestinationConversionUtils.kt 70.00% <0.00%> (-1.12%) ⬇️
...nationmigration/DestinationMigrationUtilService.kt 81.45% <0.00%> (-0.81%) ⬇️
...ing/model/destination/DestinationContextFactory.kt 75.00% <0.00%> (ø)
...rch/alerting/resthandler/RestIndexMonitorAction.kt 81.03% <0.00%> (+0.33%) ⬆️
.../kotlin/org/opensearch/alerting/util/IndexUtils.kt 72.91% <0.00%> (+2.08%) ⬆️
...alerting/transport/TransportSearchMonitorAction.kt 75.00% <0.00%> (+2.50%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Member

@getsaurabh02 getsaurabh02 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @eirsep . Overall approach looks good to me. Few comments/clarifications to consider.

Comment on lines +339 to +341
if (grouByFields > 0) {
return listOf()
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to return from here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are more groupby fields then aggregation buckets would become tuples. We can develop iteratively and first begin by supporting only for a single field(buckets with single field aggregated on) to filter on.

Comment on lines 410 to 411
monitorCtx.client!!.suspendUntil<Client, IndexResponse> {
monitorCtx.client!!.index(indexRequest, it)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we indexing each finding document, one at a time? Wondering if we should use bulk request instead?

findings.add(finding.id)
}
}
return findings
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to return the findings from here, as indexing is already done?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are returning finding ids. finding ids list is populated in the alert.

@@ -1321,6 +1323,71 @@ class MonitorRunnerServiceIT : AlertingRestTestCase() {
)
}

fun `test bucket-level monitor with findings enabled`() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets validate other scenarios as well:

  1. Where findings will not be created - such as compound aggregation.
  2. Findings index unavailable, or exception in index.

@eirsep eirsep merged commit 5b451b9 into opensearch-project:main Nov 5, 2022
@eirsep eirsep deleted the blm branch November 5, 2022 19:11
opensearch-trigger-bot bot pushed a commit that referenced this pull request Nov 5, 2022
* bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

* add test to verify bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

* added tests. fixed document ids in bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

Signed-off-by: Surya Sashank Nistala <[email protected]>
(cherry picked from commit 5b451b9)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Nov 5, 2022
* bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

* add test to verify bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

* added tests. fixed document ids in bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

Signed-off-by: Surya Sashank Nistala <[email protected]>
(cherry picked from commit 5b451b9)
eirsep added a commit that referenced this pull request Nov 6, 2022
* bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

* add test to verify bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

* added tests. fixed document ids in bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

Signed-off-by: Surya Sashank Nistala <[email protected]>
(cherry picked from commit 5b451b9)

Co-authored-by: Surya Sashank Nistala <[email protected]>
eirsep added a commit that referenced this pull request Nov 7, 2022
* bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

* add test to verify bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

* added tests. fixed document ids in bucket level monitor findings

Signed-off-by: Surya Sashank Nistala <[email protected]>

Signed-off-by: Surya Sashank Nistala <[email protected]>
(cherry picked from commit 5b451b9)

Co-authored-by: Surya Sashank Nistala <[email protected]>
@eirsep eirsep mentioned this pull request Nov 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants