Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate with Notifications plugin for Alerting backend #401

Merged
merged 10 commits into from
Apr 17, 2022

Conversation

qreshi
Copy link
Contributor

@qreshi qreshi commented Apr 13, 2022

Issue #, if available: #105

Description of changes:
This PR depends on opensearch-project/common-utils#158 and opensearch-project/notifications#406 so the GitHub Actions are expected to fail without at least common-utils being merged in

The main changes in this PR are:

  • Integrating with the Notifications plugin, supporting execution of the notification actions before/after migration of the Destination configs to Notifications
    • In the case of the Destination failing migration, it will be sent through the passthrough API (this avoids hard failures on a write block but still requires the Notification plugin to be installed since the passthrough API does not work without it)
  • Fixes some bugs
    • For the Destination migration, email_group configs were being checked with a string match which could also appear in the Email Destination (this would cause it to use the wrong parser so the fetching of the different config types has been split up)
    • The Alerting email keystore settings were not correctly declaring the fallback setting to the legacy opendistro*. variants this would cause runtime errors when it tried to load the keystore settings and failed on falling back to the old setting
      • This has been fixed (although it should not be used anymore since the Notification plugin has its own namespace for these settings that will take precedence, Alerting's will be used as a fallback for that)

CheckList:
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@qreshi qreshi requested a review from a team April 13, 2022 16:35
logger.info("Migrated ${migratedDestinations.size} destinations")
val failedDeletedDestinations = deleteOldDestinations(client, migratedDestinations)
logger.info("Failed to delete ${failedDeletedDestinations.size} destinations from migration process cleanup")
val migratedConfigs = createNotificationChannelIfNotExists(client, configsToMigrate)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be scope of this PR, but what if the user who created the notification config, is deleted? Will it still migrate it? Or will it fail and keep retrying forever?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean the user we're trying to inject in the thread context for the object is deleted?

Hmm, Security might handle that and treat it as no permissions? We'll want to test that when we test Alerting/Notif security integration. That's out of scope for this PR for now but a good callout.

// Adding a check on TEST_ACTION Destination type here to avoid supporting it as a LegacyBaseMessage type
// just for Alerting integration tests
if (config.destination?.isTestAction() == true) {
return "test action"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we define this constant somewhere else and use it both at the time assert and here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

return "test action"
}

if (config.destination?.isAllowed(allowList) == false) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we rename allowList to destinationsAllowList for better readability.

// Adding a check on TEST_ACTION Destination type here to avoid supporting it as a LegacyBaseMessage type
// just for Alerting integration tests
if (config.destination?.isTestAction() == true) {
return "test action"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@@ -85,6 +96,14 @@ class DestinationSettings {
return concreteSetting.get(settings)
}

private fun <T> fallback(key: String, affixSetting: AffixSetting<T>, regex: String, replacement: String): Setting<T>? {
return if ("_na_" == key) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

@qreshi qreshi force-pushed the monitor-exec-notif-integration branch from 76ebd85 to a997d40 Compare April 17, 2022 15:55
@codecov-commenter
Copy link

codecov-commenter commented Apr 17, 2022

Codecov Report

Merging #401 (a997d40) into main (396fc6d) will increase coverage by 0.19%.
The diff coverage is 48.09%.

@@             Coverage Diff              @@
##               main     #401      +/-   ##
============================================
+ Coverage     78.31%   78.50%   +0.19%     
+ Complexity      268      263       -5     
============================================
  Files           180      180              
  Lines          7631     7625       -6     
  Branches       1065     1069       +4     
============================================
+ Hits           5976     5986      +10     
+ Misses         1106     1083      -23     
- Partials        549      556       +7     
Impacted Files Coverage Δ
...org/opensearch/alerting/model/destination/Chime.kt 80.00% <0.00%> (+16.00%) ⬆️
...org/opensearch/alerting/model/destination/Slack.kt 80.00% <0.00%> (+16.00%) ⬆️
...destinationmigration/DestinationConversionUtils.kt 71.11% <ø> (+2.22%) ⬆️
...ing/destination/client/DestinationEmailClient.java 72.50% <ø> (-5.00%) ⬇️
...ensearch/alerting/model/destination/Destination.kt 75.60% <23.52%> (+0.46%) ⬆️
...in/kotlin/org/opensearch/alerting/MonitorRunner.kt 53.51% <37.50%> (-3.06%) ⬇️
.../util/destinationmigration/NotificationApiUtils.kt 57.53% <44.00%> (+37.53%) ⬆️
...pensearch/alerting/settings/DestinationSettings.kt 70.73% <77.77%> (+0.14%) ⬆️
...nationmigration/DestinationMigrationUtilService.kt 81.45% <85.00%> (-1.75%) ⬇️
...tlin/org/opensearch/alerting/util/AlertingUtils.kt 82.60% <100.00%> (+0.79%) ⬆️
... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 396fc6d...a997d40. Read the comment docs.

@qreshi
Copy link
Contributor Author

qreshi commented Apr 17, 2022

As discussed, the non-blocking comments will be addressed in a follow-up.

Also, the only failing GitHub Actions now that the dependent PRs are merged in is the security workflow which is being fixed in #407, merging in for now.

@qreshi qreshi merged commit 4d8643c into opensearch-project:main Apr 17, 2022
Angie-Zhang pushed a commit to Angie-Zhang/alerting that referenced this pull request Jun 29, 2022
…project#401)

* Add utils for retrieving notifications to both legacy Destinations and Notification Channels

Signed-off-by: Mohammad Qureshi <[email protected]>

* Refactor runAction in MonitorRunner to be able to send notifications to either Notification Channels or Destination

Signed-off-by: Mohammad Qureshi <[email protected]>

* Fix error handling when Notification config is not found and support TEST_ACTION for tests

Signed-off-by: Mohammad Qureshi <[email protected]>

* Fix issue with fallback setting for Destination email keystore settings

Signed-off-by: Mohammad Qureshi <[email protected]>

* Add publishing email Destinations via Notifications passthrough API

Signed-off-by: Mohammad Qureshi <[email protected]>

* Remove unused code in NotificationApiUtils

Signed-off-by: Mohammad Qureshi <[email protected]>

* Use subject as the title for Email Channels

Signed-off-by: Mohammad Qureshi <[email protected]>

* Combine runAction() methods

Signed-off-by: Mohammad Qureshi <[email protected]>

* Pass accountName to LegacyEmailMessage

Signed-off-by: Mohammad Qureshi <[email protected]>

* Split retrieval of Destination configs in DestinationMigrationUtilService

Signed-off-by: Mohammad Qureshi <[email protected]>
Signed-off-by: Angie Zhang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants