[Synthetics] adjust alert timing #159511
Merged
[Synthetics] adjust alert timing #159511
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dominiqueclarke
added
bug
|
Pinging @elastic/uptime (Team:uptime) |
🤖 GitHub commentsExpand to view the GitHub comments
Just comment with:
|
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]Unknown metric groupsESLint disabled line counts
Total ESLint disabled count
To update your PR or re-run it, just comment with: |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jun 12, 2023
### Release note Resolves elastic#158172 Fixes an issue where alerting on Synthetics monitors could sometimes become delayed. ## Summary Adjust alerting timing to account for alerting framework delays. Alert executors do not fire perfectly on the specified interval (in our case, every 1 minute). They are typically delayed a few seconds, be can especially delayed if the customer does not have their alerting settings configured to scale and there is no CPU available to take on the scheduled task. This PR accounts for small delays in the alerting framework, by subtracting an additional 1 minute from the `from` value for our current status query. (cherry picked from commit d549500)
kibanamachine
added a commit
that referenced
this pull request
Jun 12, 2023
# Backport This will backport the following commits from `main` to `8.8`: - [[Synthetics] adjust alert timing (#159511)](#159511) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Dominique Clarke","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-06-12T19:08:53Z","message":"[Synthetics] adjust alert timing (#159511)\n\n### Release note\r\n\r\nResolves https://github.com/elastic/kibana/issues/158172\r\n\r\nFixes an issue where alerting on Synthetics monitors could sometimes\r\nbecome delayed.\r\n \r\n## Summary\r\nAdjust alerting timing to account for alerting framework delays.\r\n\r\nAlert executors do not fire perfectly on the specified interval (in our\r\ncase, every 1 minute). They are typically delayed a few seconds, be can\r\nespecially delayed if the customer does not have their alerting settings\r\nconfigured to scale and there is no CPU available to take on the\r\nscheduled task.\r\n\r\nThis PR accounts for small delays in the alerting framework, by\r\nsubtracting an additional 1 minute from the `from` value for our current\r\nstatus query.","sha":"d549500d3433b67cd5ec1f7aa60e866d4fb699c1","branchLabelMapping":{"^v8.9.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["bug","release_note:fix","Team:uptime","v8.9.0","v8.8.2"],"number":159511,"url":"https://github.com/elastic/kibana/pull/159511","mergeCommit":{"message":"[Synthetics] adjust alert timing (#159511)\n\n### Release note\r\n\r\nResolves https://github.com/elastic/kibana/issues/158172\r\n\r\nFixes an issue where alerting on Synthetics monitors could sometimes\r\nbecome delayed.\r\n \r\n## Summary\r\nAdjust alerting timing to account for alerting framework delays.\r\n\r\nAlert executors do not fire perfectly on the specified interval (in our\r\ncase, every 1 minute). They are typically delayed a few seconds, be can\r\nespecially delayed if the customer does not have their alerting settings\r\nconfigured to scale and there is no CPU available to take on the\r\nscheduled task.\r\n\r\nThis PR accounts for small delays in the alerting framework, by\r\nsubtracting an additional 1 minute from the `from` value for our current\r\nstatus query.","sha":"d549500d3433b67cd5ec1f7aa60e866d4fb699c1"}},"sourceBranch":"main","suggestedTargetBranches":["8.8"],"targetPullRequestStates":[{"branch":"main","label":"v8.9.0","labelRegex":"^v8.9.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/159511","number":159511,"mergeCommit":{"message":"[Synthetics] adjust alert timing (#159511)\n\n### Release note\r\n\r\nResolves https://github.com/elastic/kibana/issues/158172\r\n\r\nFixes an issue where alerting on Synthetics monitors could sometimes\r\nbecome delayed.\r\n \r\n## Summary\r\nAdjust alerting timing to account for alerting framework delays.\r\n\r\nAlert executors do not fire perfectly on the specified interval (in our\r\ncase, every 1 minute). They are typically delayed a few seconds, be can\r\nespecially delayed if the customer does not have their alerting settings\r\nconfigured to scale and there is no CPU available to take on the\r\nscheduled task.\r\n\r\nThis PR accounts for small delays in the alerting framework, by\r\nsubtracting an additional 1 minute from the `from` value for our current\r\nstatus query.","sha":"d549500d3433b67cd5ec1f7aa60e866d4fb699c1"}},{"branch":"8.8","label":"v8.8.2","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Dominique Clarke <[email protected]>
jloleysens
added a commit
to jloleysens/kibana
that referenced
this pull request
Jun 13, 2023
* main: (199 commits) [Lens] Add custom formatter within the Lens editor (elastic#158468) [Uptime] Hide app if no data is available (elastic#159118) [CodeEditor] Add grok highlighting (elastic#159334) Expose decoded cloudId components from the cloud plugin's contract (elastic#159442) [Profiling] Use collector and symbolizer integrations in the add data page (elastic#159481) [Infrastructure UI] Hosts View: Unified Search bar with auto-refresh enabled (elastic#157011) [APM] Add feature flag for not available apm schema (elastic#158911) [Lens] Remove deprecated componentWillReceiveProps usage (elastic#159502) [api-docs] 2023-06-13 Daily api_docs build (elastic#159536) [data views] Use versioned router for REST routes (elastic#158608) [maps] fix geo line source not loaded unless maps application is opened (elastic#159432) [Enterprise Search][Search application]Fix Create Api key url (elastic#159519) [Security Solution] Increase timeout for indexing hosts (elastic#159518) dashboard Reset button disable (elastic#159430) [Security Solution] Unskip Endpoint API tests after package fix (elastic#159484) [Synthetics] adjust alert timing (elastic#159511) [ResponseOps][rule registry] Remove usages of `refresh: true` (elastic#159252) Revert "Remove unused package (elastic#158597)" [Serverless] Adding config to disable authentication on task manager background worker utilization API (elastic#159505) Remove unused package (elastic#158597) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release note
Resolves #158172
Fixes an issue where alerting on Synthetics monitors could sometimes become delayed.
Summary
Adjust alerting timing to account for alerting framework delays.
Alert executors do not fire perfectly on the specified interval (in our case, every 1 minute). They are typically delayed a few seconds, be can especially delayed if the customer does not have their alerting settings configured to scale and there is no CPU available to take on the scheduled task.
This PR accounts for small delays in the alerting framework, by subtracting an additional 1 minute from the
fromvalue for our current status query.