Implemented cross-cluster monitors. #871

AWSHurneyt · 2024-02-02T08:38:05Z

Description

Added support for configuring query, bucket, and cluster metrics monitors to execute against remote clusters.

Documentation PR for reference opensearch-project/documentation-website#6350

Issues Resolved

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

codecov · 2024-02-02T08:51:26Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (30b3b71) 49.85% compared to head (0b232d3) 48.69%.
Report is 11 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #871      +/-   ##
==========================================
- Coverage   49.85%   48.69%   -1.17%     
==========================================
  Files         233      236       +3     
  Lines        6561     6845     +284     
  Branches      939      996      +57     
==========================================
+ Hits         3271     3333      +62     
- Misses       3287     3509     +222     
  Partials        3        3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

* Support any channel types from Notification Signed-off-by: Ashish Agrawal <[email protected]> * Remove unused CHANNEL_TYPES constant Signed-off-by: Ashish Agrawal <[email protected]> * Return empty array if failed to get features Signed-off-by: Ashish Agrawal <[email protected]> --------- Signed-off-by: Ashish Agrawal <[email protected]> Signed-off-by: AWSHurneyt <[email protected]>

* Drafted 2.11 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Drafted 2.11 release notes. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

* Removed "last updated by" section from the UI as the SearchMonitor API can no longer return that info. Signed-off-by: AWSHurneyt <[email protected]> * Updated cypress workflow. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

…ct#789) * Onboard Jenkins prod docker image to github actions Signed-off-by: Peter Zhu <[email protected]> * Small typos Signed-off-by: Peter Zhu <[email protected]> * Add back workflows Signed-off-by: Peter Zhu <[email protected]> * Restore macos/windows Signed-off-by: Peter Zhu <[email protected]> --------- Signed-off-by: Peter Zhu <[email protected]> Signed-off-by: AWSHurneyt <[email protected]>

* Bumped babel version. Adjusted babel config. Refactored unit test that started failing after bump. Signed-off-by: AWSHurneyt <[email protected]> * Updated yarn file. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

* fixed incorrect use of this.props Signed-off-by: Amardeepsingh Siglani <[email protected]> * resolved dependency conflict Signed-off-by: Amardeepsingh Siglani <[email protected]> * updated babel config Signed-off-by: Amardeepsingh Siglani <[email protected]> --------- Signed-off-by: Amardeepsingh Siglani <[email protected]> Signed-off-by: AWSHurneyt <[email protected]>

* Amended 2.11.0 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Added 2.11.1 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Added 2.11.1 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Added 2.11.1 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Added 2.11.1 release notes. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

…ject#827) * Fixed a bug that was causing groupBy/aggregation fields from displaying in various areas of the UI. Related issues: 816, 817, 818. Signed-off-by: AWSHurneyt <[email protected]> * Fixed trigger context object bug in issue 791. Signed-off-by: AWSHurneyt <[email protected]> * Capitalized bucket column titles, and moved bucket columns to the end of the column array. Signed-off-by: AWSHurneyt <[email protected]> * Added wait steps to reduce test flakiness. Signed-off-by: AWSHurneyt <[email protected]> * Added wait step to reduce test flakiness. Adjusted test monitor trigger condition to always triggers on a healthy clusters. Signed-off-by: AWSHurneyt <[email protected]> * Removed unused imports. Signed-off-by: AWSHurneyt <[email protected]> * fixed bucket level monitor flaky cypress test Signed-off-by: Amardeepsingh Siglani <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]> Signed-off-by: Amardeepsingh Siglani <[email protected]> Co-authored-by: Amardeepsingh Siglani <[email protected]> Signed-off-by: AWSHurneyt <[email protected]>

…project#794) Signed-off-by: Chenxi Wang <[email protected]> Signed-off-by: AWSHurneyt <[email protected]>

Signed-off-by: Derek Ho <[email protected]> Signed-off-by: AWSHurneyt <[email protected]>

…ect#854) Signed-off-by: Amardeepsingh Siglani <[email protected]> Signed-off-by: AWSHurneyt <[email protected]>

Signed-off-by: AWSHurneyt <[email protected]>

…, and cluster metrics monitors. Signed-off-by: AWSHurneyt <[email protected]>

…in value. Signed-off-by: AWSHurneyt <[email protected]>

…itor details page to show data sources. Signed-off-by: AWSHurneyt <[email protected]>

Signed-off-by: AWSHurneyt <[email protected]>

AWSHurneyt · 2024-02-05T20:55:35Z

This PR depends on the backend changes.
opensearch-project/alerting#1404

…monitors when remote monitoring is disabled. Signed-off-by: AWSHurneyt <[email protected]>

Signed-off-by: AWSHurneyt <[email protected]>

AWSHurneyt · 2024-02-06T00:37:08Z

Only the cluster_metrics_monitor_spec cypress tests are failing; however, they're passing locally. Troubleshooting what could be causing the discrepancy.

Signed-off-by: AWSHurneyt <[email protected]>

amsiglan · 2024-02-06T07:18:39Z

public/components/Flyout/flyouts/components/AlertsDashboardFlyoutComponent.js

@@ -150,6 +156,12 @@ export default class AlertsDashboardFlyoutComponent extends Component {
    }
  };

+  getLocalClusterName = async () => {


Nit: Not a blocker but would rename this to something like setLocalClusterName since we are not returning anything in here

amsiglan · 2024-02-06T07:21:04Z

...mponents/Flyout/flyouts/components/__snapshots__/AlertsDashboardFlyoutComponent.test.js.snap

+            }
+          }
+        >
+          -


Nit: DAEFAULT_EMPTY_DATA here?
not a blocker

* Support any channel types from Notification (#743) * Support any channel types from Notification Signed-off-by: Ashish Agrawal <[email protected]> * Remove unused CHANNEL_TYPES constant Signed-off-by: Ashish Agrawal <[email protected]> * Return empty array if failed to get features Signed-off-by: Ashish Agrawal <[email protected]> --------- Signed-off-by: Ashish Agrawal <[email protected]> Signed-off-by: AWSHurneyt <[email protected]> * Drafted 2.11 release notes. (#764) * Drafted 2.11 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Drafted 2.11 release notes. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]> * Removed "last updated by" sections from the UI. (#767) * Removed "last updated by" section from the UI as the SearchMonitor API can no longer return that info. Signed-off-by: AWSHurneyt <[email protected]> * Updated cypress workflow. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]> * Onboard Jenkins prod docker image to github actions (#789) * Onboard Jenkins prod docker image to github actions Signed-off-by: Peter Zhu <[email protected]> * Small typos Signed-off-by: Peter Zhu <[email protected]> * Add back workflows Signed-off-by: Peter Zhu <[email protected]> * Restore macos/windows Signed-off-by: Peter Zhu <[email protected]> --------- Signed-off-by: Peter Zhu <[email protected]> Signed-off-by: AWSHurneyt <[email protected]> * Bumped babel version. (#821) * Bumped babel version. Adjusted babel config. Refactored unit test that started failing after bump. Signed-off-by: AWSHurneyt <[email protected]> * Updated yarn file. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]> * Fix fetching of channels for composite monitors (#820) * fixed incorrect use of this.props Signed-off-by: Amardeepsingh Siglani <[email protected]> * resolved dependency conflict Signed-off-by: Amardeepsingh Siglani <[email protected]> * updated babel config Signed-off-by: Amardeepsingh Siglani <[email protected]> --------- Signed-off-by: Amardeepsingh Siglani <[email protected]> Signed-off-by: AWSHurneyt <[email protected]> * Added 2.11.1 release notes. (#828) * Amended 2.11.0 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Added 2.11.1 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Added 2.11.1 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Added 2.11.1 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Added 2.11.1 release notes. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]> * Fixed bucket monitor groupBy/aggregation display bug. (#827) * Fixed a bug that was causing groupBy/aggregation fields from displaying in various areas of the UI. Related issues: 816, 817, 818. Signed-off-by: AWSHurneyt <[email protected]> * Fixed trigger context object bug in issue 791. Signed-off-by: AWSHurneyt <[email protected]> * Capitalized bucket column titles, and moved bucket columns to the end of the column array. Signed-off-by: AWSHurneyt <[email protected]> * Added wait steps to reduce test flakiness. Signed-off-by: AWSHurneyt <[email protected]> * Added wait step to reduce test flakiness. Adjusted test monitor trigger condition to always triggers on a healthy clusters. Signed-off-by: AWSHurneyt <[email protected]> * Removed unused imports. Signed-off-by: AWSHurneyt <[email protected]> * fixed bucket level monitor flaky cypress test Signed-off-by: Amardeepsingh Siglani <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]> Signed-off-by: Amardeepsingh Siglani <[email protected]> Co-authored-by: Amardeepsingh Siglani <[email protected]> Signed-off-by: AWSHurneyt <[email protected]> * Issue #671 fix trigger name validation (#794) Signed-off-by: Chenxi Wang <[email protected]> Signed-off-by: AWSHurneyt <[email protected]> * Remove integtest.sh since it is not being used (#849) Signed-off-by: Derek Ho <[email protected]> Signed-off-by: AWSHurneyt <[email protected]> * do not create Message component on every text change (#854) Signed-off-by: Amardeepsingh Siglani <[email protected]> Signed-off-by: AWSHurneyt <[email protected]> * Implemented server API call to feature backend API. Signed-off-by: AWSHurneyt <[email protected]> * Implemented remote cluster support for creating/editing query, bucket, and cluster metrics monitors. Signed-off-by: AWSHurneyt <[email protected]> * Implemented warning model when monitor execution time exceeds a certain value. Signed-off-by: AWSHurneyt <[email protected]> * Updated alert details flyout to show remote cluster info. Updated monitor details page to show data sources. Signed-off-by: AWSHurneyt <[email protected]> * Updated unit tests. Signed-off-by: AWSHurneyt <[email protected]> * Added experimental banner. Signed-off-by: AWSHurneyt <[email protected]> * Updated snapshots. Signed-off-by: AWSHurneyt <[email protected]> * Edited text on the experimental banner. Signed-off-by: AWSHurneyt <[email protected]> * Moved getSettings call to hide Data source panel for cluster metrics monitors when remote monitoring is disabled. Signed-off-by: AWSHurneyt <[email protected]> * Updated snapshots. Signed-off-by: AWSHurneyt <[email protected]> * Increased cypress test timeout. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: Ashish Agrawal <[email protected]> Signed-off-by: AWSHurneyt <[email protected]> Signed-off-by: Peter Zhu <[email protected]> Signed-off-by: Amardeepsingh Siglani <[email protected]> Signed-off-by: Chenxi Wang <[email protected]> Signed-off-by: Derek Ho <[email protected]> Co-authored-by: Ashish Agrawal <[email protected]> Co-authored-by: Peter Zhu <[email protected]> Co-authored-by: Amardeepsingh Siglani <[email protected]> Co-authored-by: Chenxi Wang <[email protected]> Co-authored-by: Derek Ho <[email protected]> (cherry picked from commit fb82368) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Support any channel types from Notification (#743) * Support any channel types from Notification * Remove unused CHANNEL_TYPES constant * Return empty array if failed to get features --------- * Drafted 2.11 release notes. (#764) * Drafted 2.11 release notes. * Drafted 2.11 release notes. --------- * Removed "last updated by" sections from the UI. (#767) * Removed "last updated by" section from the UI as the SearchMonitor API can no longer return that info. * Updated cypress workflow. --------- * Onboard Jenkins prod docker image to github actions (#789) * Onboard Jenkins prod docker image to github actions * Small typos * Add back workflows * Restore macos/windows --------- * Bumped babel version. (#821) * Bumped babel version. Adjusted babel config. Refactored unit test that started failing after bump. * Updated yarn file. --------- * Fix fetching of channels for composite monitors (#820) * fixed incorrect use of this.props * resolved dependency conflict * updated babel config --------- * Added 2.11.1 release notes. (#828) * Amended 2.11.0 release notes. * Added 2.11.1 release notes. * Added 2.11.1 release notes. * Added 2.11.1 release notes. * Added 2.11.1 release notes. --------- * Fixed bucket monitor groupBy/aggregation display bug. (#827) * Fixed a bug that was causing groupBy/aggregation fields from displaying in various areas of the UI. Related issues: 816, 817, 818. * Fixed trigger context object bug in issue 791. * Capitalized bucket column titles, and moved bucket columns to the end of the column array. * Added wait steps to reduce test flakiness. * Added wait step to reduce test flakiness. Adjusted test monitor trigger condition to always triggers on a healthy clusters. * Removed unused imports. * fixed bucket level monitor flaky cypress test --------- * Issue #671 fix trigger name validation (#794) * Remove integtest.sh since it is not being used (#849) * do not create Message component on every text change (#854) * Implemented server API call to feature backend API. * Implemented remote cluster support for creating/editing query, bucket, and cluster metrics monitors. * Implemented warning model when monitor execution time exceeds a certain value. * Updated alert details flyout to show remote cluster info. Updated monitor details page to show data sources. * Updated unit tests. * Added experimental banner. * Updated snapshots. * Edited text on the experimental banner. * Moved getSettings call to hide Data source panel for cluster metrics monitors when remote monitoring is disabled. * Updated snapshots. * Increased cypress test timeout. --------- (cherry picked from commit fb82368) Signed-off-by: Ashish Agrawal <[email protected]> Signed-off-by: AWSHurneyt <[email protected]> Signed-off-by: Peter Zhu <[email protected]> Signed-off-by: Amardeepsingh Siglani <[email protected]> Signed-off-by: Chenxi Wang <[email protected]> Signed-off-by: Derek Ho <[email protected]> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Ashish Agrawal <[email protected]> Co-authored-by: Peter Zhu <[email protected]> Co-authored-by: Amardeepsingh Siglani <[email protected]> Co-authored-by: Chenxi Wang <[email protected]> Co-authored-by: Derek Ho <[email protected]>

AWSHurneyt added the backport 2.x label Feb 2, 2024

AWSHurneyt force-pushed the 3.0-cross-cluster-dev branch from 4b353f1 to caa1338 Compare February 5, 2024 18:02

lezzago and others added 19 commits February 5, 2024 12:45

Drafted 2.11 release notes. (opensearch-project#764)

c94ed8b

* Drafted 2.11 release notes. Signed-off-by: AWSHurneyt <[email protected]> * Drafted 2.11 release notes. Signed-off-by: AWSHurneyt <[email protected]> --------- Signed-off-by: AWSHurneyt <[email protected]>

Issue opensearch-project#671 fix trigger name validation (opensearch-…

63166b9

…project#794) Signed-off-by: Chenxi Wang <[email protected]> Signed-off-by: AWSHurneyt <[email protected]>

Remove integtest.sh since it is not being used (opensearch-project#849)

2025162

Signed-off-by: Derek Ho <[email protected]> Signed-off-by: AWSHurneyt <[email protected]>

do not create Message component on every text change (opensearch-proj…

9a61a06

…ect#854) Signed-off-by: Amardeepsingh Siglani <[email protected]> Signed-off-by: AWSHurneyt <[email protected]>

Implemented server API call to feature backend API.

6c0c862

Signed-off-by: AWSHurneyt <[email protected]>

Implemented remote cluster support for creating/editing query, bucket…

f05bee9

…, and cluster metrics monitors. Signed-off-by: AWSHurneyt <[email protected]>

Implemented warning model when monitor execution time exceeds a certa…

9ba60fe

…in value. Signed-off-by: AWSHurneyt <[email protected]>

Updated alert details flyout to show remote cluster info. Updated mon…

0c48037

…itor details page to show data sources. Signed-off-by: AWSHurneyt <[email protected]>

Updated unit tests.

4a986ed

Signed-off-by: AWSHurneyt <[email protected]>

Added experimental banner.

d18bcfc

Signed-off-by: AWSHurneyt <[email protected]>

Updated snapshots.

de361c0

Signed-off-by: AWSHurneyt <[email protected]>

Edited text on the experimental banner.

fb41a75

Signed-off-by: AWSHurneyt <[email protected]>

AWSHurneyt force-pushed the 3.0-cross-cluster-dev branch from 52ead58 to fb41a75 Compare February 5, 2024 20:51

AWSHurneyt added 2 commits February 5, 2024 15:29

Moved getSettings call to hide Data source panel for cluster metrics …

9dd22cc

…monitors when remote monitoring is disabled. Signed-off-by: AWSHurneyt <[email protected]>

Updated snapshots.

6e75711

Signed-off-by: AWSHurneyt <[email protected]>

AWSHurneyt marked this pull request as ready for review February 5, 2024 23:42

AWSHurneyt requested review from lezzago, amsiglan and sbcd90 as code owners February 5, 2024 23:43

AWSHurneyt requested review from eirsep, getsaurabh02, praveensameneni, qreshi, bowenlan-amzn and rishabhmaurya as code owners February 5, 2024 23:43

Increased cypress test timeout.

0b232d3

Signed-off-by: AWSHurneyt <[email protected]>

praveensameneni approved these changes Feb 6, 2024

View reviewed changes

amsiglan reviewed Feb 6, 2024

View reviewed changes

amsiglan approved these changes Feb 6, 2024

View reviewed changes

amsiglan merged commit fb82368 into opensearch-project:main Feb 6, 2024
8 of 10 checks passed

opensearch-trigger-bot bot mentioned this pull request Feb 6, 2024

[Backport 2.x] Implemented cross-cluster monitors. #872

Merged

AWSHurneyt mentioned this pull request Feb 14, 2024

[AUTOCUT] Integration Test failed for alertingDashboards: 2.12.0 tar distribution #867

Closed

AWSHurneyt deleted the 3.0-cross-cluster-dev branch April 12, 2024 18:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implemented cross-cluster monitors. #871

Implemented cross-cluster monitors. #871

AWSHurneyt commented Feb 2, 2024 •

edited

Loading

codecov bot commented Feb 2, 2024 •

edited

Loading

AWSHurneyt commented Feb 5, 2024

AWSHurneyt commented Feb 6, 2024

amsiglan Feb 6, 2024

amsiglan Feb 6, 2024

Implemented cross-cluster monitors. #871

Implemented cross-cluster monitors. #871

Conversation

AWSHurneyt commented Feb 2, 2024 • edited Loading

Description

Issues Resolved

Check List

codecov bot commented Feb 2, 2024 • edited Loading

Codecov Report

AWSHurneyt commented Feb 5, 2024

AWSHurneyt commented Feb 6, 2024

amsiglan Feb 6, 2024

Choose a reason for hiding this comment

amsiglan Feb 6, 2024

Choose a reason for hiding this comment

AWSHurneyt commented Feb 2, 2024 •

edited

Loading

codecov bot commented Feb 2, 2024 •

edited

Loading