Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[responseOps] add snoozing MVP in alerting framework #127694

Merged
merged 6 commits into from
Mar 25, 2022

Conversation

pmuellr
Copy link
Member

@pmuellr pmuellr commented Mar 15, 2022

Summary

resolves #126512

Changes the calculation of rule-level muting to take into account snoozeEndTime.

If muteAll is true, the rule is considered snoozing forever, regardless of the setting of snoozeEndTime.

If muteAll is false, snoozeEndTime determines whether the rule is snoozed. If snoozeEndTime is null, the rule is not snoozed. Otherwise, snoozeEndTime is a Date, and if it's >= than Date.now(), the rule is snoozed. Otherwise, the rule is not snoozed.

this PR branch was originally based off of #127081

testing locally

To test, create a rule that is firing all the time that you can check - using server log is usually the easiest. Use a 1s interval so you will see the effect quicker.

To set a snooze, use the following curl command:

curl https://elastic:changeme@localhost:5601/internal/alerting/rule/${RULE_ID}/_snooze \
   -H "kbn-xsrf:foo" \
   -H "content-type: application/json" \
   -d '{"snooze_end_time": "2022-03-22T16:22:45"}'

Note the time value should be an ISO date - using a Z at the end implies UTC, without a Z, assumes local.

You can also use the value -1 (not a string) which functions the same as "mute all".

To "unsnooze", currently you have to use "unmute all":

curl https://elastic:changeme@localhost:5601/api/alerting/rule/${RULE_ID}/_unmute_all \
    -H "kbn-xsrf:foo" \
    -X POST

Checklist

Delete any items that are not applicable to this PR.

@pmuellr pmuellr force-pushed the alerting/126512-snooze-impl branch 2 times, most recently from 99f8ce6 to e45b79f Compare March 20, 2022 03:12
@pmuellr pmuellr added release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Feature:Alerting/RulesFramework Issues related to the Alerting Rules Framework v8.2.0 labels Mar 22, 2022
resolves elastic#126512

Changes the calculation of rule-level muting to take into account
snoozeEndTime.

If muteAll is true, the rule is considered snoozing forever, regardless
of the setting of snoozeEndTime.

If muteAll is false, snoozeEndTime determines whether the rule is
snoozed.  If snoozeEndTime is null, the rule is not snoozed.  Otherwise,
snoozeEndTime is a Date, and if it's >= than Date.now(), the rule
is snoozed.  Otherwise, the rule is not snoozed.
@pmuellr pmuellr force-pushed the alerting/126512-snooze-impl branch from 045937d to 99f5e68 Compare March 23, 2022 03:44
@pmuellr pmuellr marked this pull request as ready for review March 23, 2022 12:44
@pmuellr pmuellr requested a review from a team as a code owner March 23, 2022 12:44
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

Copy link
Contributor

@ymao1 ymao1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Nice job :)

Verified by:

  • creating an Always Firing rule with server log action
  • snoozing via API with snooze_end_time and verifying that no actions were written until snooze_end_time has passed
  • snoozing via API with snooze_end_time = -1 and verifying that no actions were written until setting unmuting via API

x-pack/plugins/alerting/server/task_runner/task_runner.ts Outdated Show resolved Hide resolved
@pmuellr
Copy link
Member Author

pmuellr commented Mar 24, 2022

@elasticmachine merge upstream

Copy link
Contributor

@XavierM XavierM left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the way you did it, simplicity === hard

@mikecote mikecote self-requested a review March 24, 2022 17:38
Copy link
Contributor

@mikecote mikecote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM! Tested locally and saw the notifications stop until the end time 👍

expect(connStatus).to.be(200);
objectRemover.add(Spaces.space1.id, createdAction.id, 'action', 'actions');

log.info('creating rule');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: should these logs be info or debug? I believe CI used to or still captures the debug logs whenever a flaky or failed test happens.

@pmuellr
Copy link
Member Author

pmuellr commented Mar 24, 2022

@elasticmachine merge upstream

@pmuellr
Copy link
Member Author

pmuellr commented Mar 25, 2022

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting Feature:Alerting/RulesFramework Issues related to the Alerting Rules Framework release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v8.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[E&C] add snoozing MVP in alerting framework
7 participants