-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Alerting] Skip writing alerts when rule execution times out #114518
[Alerting] Skip writing alerts when rule execution times out #114518
Conversation
…ution is cancelled
…ing/cancel-alerting-tasks
…ing/cancel-alerting-tasks
423695a
to
56fcff0
Compare
56fcff0
to
7fac6a2
Compare
…ing/cancel-alerting-tasks
…ing/cancel-alerting-tasks
…ing/cancel-alerting-tasks
…ing-aad-on-timeout-2
💚 Build SucceededMetrics [docs]Public APIs missing comments
Async chunks
Page load bundle
History
To update your PR or re-run it, just comment with: |
…g/skip-writing-aad-on-timeout-2
…g/skip-writing-aad-on-timeout-2
…g/skip-writing-aad-on-timeout-2
@elasticmachine merge upstream |
Pinging @elastic/kibana-alerting-services (Team:Alerting Services) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
Pinging @elastic/apm-ui (Team:apm) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
@elasticmachine merge upstream |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Persistence rule type changes LGTM
@elasticmachine merge upstream |
💚 Build Succeeded
Metrics [docs]Public APIs missing comments
History
To update your PR or re-run it, just comment with: cc @ymao1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
APM changes LGTM
…#114518) * Added cancel() to alerting task runner and writing event log document * Updating rule saved object with timeout execution status * Skip scheduling actions and logging event log for alerts if rule execution is cancelled * Adding config for disabling skipping actions * Fixing types * Adding flag for rule types to opt out of skipping acitons * Passing function into rule executor to determine whether to write AAD * Using task runner uuid to differentiate between task instances * Adding functional test * Fixing types * Updating service function name and adding logic to persistence rule type Co-authored-by: Kibana Machine <[email protected]>
Resolves #113804
Summary
Followup to Alerting cancel implementation PR. In that PR, we added a
cancelAlertsOnRuleTimeout
flag that can be set at a kibana level (inkibana.yml
) or on a rule type level, where, if the flag is set totrue
for both, the alerting task runner will not schedule actions for a rule that has exceeded its timeout value.This PR extends that logic to alerts as data by passing in a
shouldWriteAlerts
service function into the rule executor that the rule registry executors (currently lifecycle and persistence) check before writing out alerts as data. In the future, I believe we can remove the need for this service function if/when we absorb the rule registry into the framework.If, as a rule type producer, you want to always write out alerts as data, regardless of whether rule execution times out, you will need to set the
cancelAlertsOnRuleTimeout
flag on the rule type tofalse
.Checklist