Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Jest Integration Tests.x-pack/plugins/security_solution/server/integration_tests - telemetry tasks telemetry-prebuilt-rule-alerts should manage runtime errors searching endpoint metrics #188234

Closed
kibanamachine opened this issue Jul 12, 2024 · 24 comments · Fixed by #190044
Closed

Failing test: Jest Integration Tests.x-pack/plugins/security_solution/server/integration_tests - telemetry tasks telemetry-prebuilt-rule-alerts should manage runtime errors searching endpoint metrics #188234

kibanamachine opened this issue Jul 12, 2024 · 24 comments · Fixed by #190044
Assignees
@szaffarano
Labels
failed-es-promotion failed-test A test failure on a tracked branch, potentially flaky-test Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Jul 12, 2024 •
edited
Loading

A test failed on a tracked branch

Error: expect(received).toBe(expected) // Object.is equality

Expected: 1
Received: 0
    at Object.toBe (/opt/buildkite-agent/builds/bk-agent-prod-gcp-1720791056497287678/elastic/kibana-elasticsearch-snapshot-verify/kibana/x-pack/plugins/security_solution/server/integration_tests/telemetry.test.ts:705:31)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)

First failure: kibana-elasticsearch-snapshot-verify - main
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Jul 12, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Jul 12, 2024
@kibanamachine kibanamachine added failed-es-promotion Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. labels Jul 12, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Jul 12, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@delanni
Copy link
Contributor

delanni commented Jul 15, 2024

/skip

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

delanni added a commit that referenced this issue Jul 15, 2024
@delanni
skip flaky test: #188234
ee35fd4
@delanni delanni mentioned this issue Jul 15, 2024
Merged
delanni added a commit that referenced this issue Jul 15, 2024
@delanni
[ci] skip flaky test (#188313)
d5345e4 
## Summary
Re: #188234
@delanni
Copy link
Contributor

delanni commented Jul 15, 2024

skipped in: #188313

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - main

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.15

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.15

@mistic mistic added the v8.15.0 label Jul 15, 2024
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Jul 15, 2024
@delanni
[ci] skip flaky test (elastic#188313)
045254c 
## Summary
Re: elastic#188234
(cherry picked from commit d5345e4)
mistic referenced this issue Jul 15, 2024
@kibanamachine @delanni
[8.15] [ci] skip flaky test (#188313) (#188381)
cc7df3d 
# Backport

This will backport the following commits from `main` to `8.15`:
- [[ci] skip flaky test
(#188313)](#188313)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Alex
Szabo","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-07-15T14:33:15Z","message":"[ci]
skip flaky test (#188313)\n\n## Summary\r\nRe:
https://github.com/elastic/kibana/issues/188234","sha":"d5345e442dce28b0ee5ef057988b156ba78784ad","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","skip-ci","backport:prev-minor","v8.16.0"],"title":"[ci]
skip flaky
test","number":188313,"url":"https://github.com/elastic/kibana/pull/188313","mergeCommit":{"message":"[ci]
skip flaky test (#188313)\n\n## Summary\r\nRe:
https://github.com/elastic/kibana/issues/188234","sha":"d5345e442dce28b0ee5ef057988b156ba78784ad"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/188313","number":188313,"mergeCommit":{"message":"[ci]
skip flaky test (#188313)\n\n## Summary\r\nRe:
https://github.com/elastic/kibana/issues/188234","sha":"d5345e442dce28b0ee5ef057988b156ba78784ad"}}]}]
BACKPORT-->

Co-authored-by: Alex Szabo <[email protected]>
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.14

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.14

mistic pushed a commit to mistic/kibana that referenced this issue Jul 16, 2024
@delanni
[ci] skip flaky test (elastic#188313)
f7b4489 
## Summary
Re: elastic#188234
(cherry picked from commit d5345e4)
@kibanamachine
Copy link
Contributor Author

New failure: kibana-elasticsearch-serverless-verify-and-promote - main

mistic added a commit that referenced this issue Jul 22, 2024
@mistic
skip flaky suite (#188234)
e33f010
@mistic
Copy link
Member

mistic commented Jul 22, 2024 •
edited
Loading

Skipped.

main: e33f010
8.15: 14af905
8.14: 33d75ff

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.15

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.15

mistic added a commit that referenced this issue Jul 24, 2024
@mistic
skip flaky suite (#188234)
0405cc7
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.14

@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.14

mistic added a commit that referenced this issue Jul 30, 2024
@mistic
skip flaky suite (#188234)
33d75ff
@szaffarano szaffarano mentioned this issue Aug 7, 2024
Merged
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Aug 7, 2024
@szaffarano
[Telemetry][Security Solution] Fix flaky tests (elastic#190044)
e0ca564 
## Summary

Fixes: elastic#188234
elastic#187719 and
elastic#178918

The flakiness was while calculating the Detection Rules task
invocations. It could have two different RCs: 1) The code didn't retry
in case the task wasn't executed yet, which makes sense in a CI
environment, which is slower than a dev environment; 2) The timestamp to
filter out requests was calculated after the task was triggered, and if
the task is executed fast enough, it could lead to empty responses
because of that.

(cherry picked from commit 358e104)
kibanamachine referenced this issue Aug 7, 2024
@kibanamachine @szaffarano
[8.15] [Telemetry][Security Solution] Fix flaky tests (#190044) (#190051
cd0bad9 
)

# Backport

This will backport the following commits from `main` to `8.15`:
- [[Telemetry][Security Solution] Fix flaky tests
(#190044)](#190044)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Sebastián
Zaffarano","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-08-07T13:13:13Z","message":"[Telemetry][Security
Solution] Fix flaky tests (#190044)\n\n## Summary\r\n\r\nFixes:
https://github.com/elastic/kibana/issues/188234\r\nhttps://github.com/elastic/kibana/issues/187719
and\r\nhttps://github.com//issues/178918\r\n\r\nThe
flakiness was while calculating the Detection Rules task\r\ninvocations.
It could have two different RCs: 1) The code didn't retry\r\nin case the
task wasn't executed yet, which makes sense in a CI\r\nenvironment,
which is slower than a dev environment; 2) The timestamp to\r\nfilter
out requests was calculated after the task was triggered, and if\r\nthe
task is executed fast enough, it could lead to empty
responses\r\nbecause of
that.","sha":"358e104ebcc2bfcfeec3457b2e913d071d27745d","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:
SecuritySolution","v8.15.0","v8.16.0"],"title":"[Telemetry][Security
Solution] Fix flaky
tests","number":190044,"url":"https://github.com/elastic/kibana/pull/190044","mergeCommit":{"message":"[Telemetry][Security
Solution] Fix flaky tests (#190044)\n\n## Summary\r\n\r\nFixes:
https://github.com/elastic/kibana/issues/188234\r\nhttps://github.com/elastic/kibana/issues/187719
and\r\nhttps://github.com//issues/178918\r\n\r\nThe
flakiness was while calculating the Detection Rules task\r\ninvocations.
It could have two different RCs: 1) The code didn't retry\r\nin case the
task wasn't executed yet, which makes sense in a CI\r\nenvironment,
which is slower than a dev environment; 2) The timestamp to\r\nfilter
out requests was calculated after the task was triggered, and if\r\nthe
task is executed fast enough, it could lead to empty
responses\r\nbecause of
that.","sha":"358e104ebcc2bfcfeec3457b2e913d071d27745d"}},"sourceBranch":"main","suggestedTargetBranches":["8.15"],"targetPullRequestStates":[{"branch":"8.15","label":"v8.15.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/190044","number":190044,"mergeCommit":{"message":"[Telemetry][Security
Solution] Fix flaky tests (#190044)\n\n## Summary\r\n\r\nFixes:
https://github.com/elastic/kibana/issues/188234\r\nhttps://github.com/elastic/kibana/issues/187719
and\r\nhttps://github.com//issues/178918\r\n\r\nThe
flakiness was while calculating the Detection Rules task\r\ninvocations.
It could have two different RCs: 1) The code didn't retry\r\nin case the
task wasn't executed yet, which makes sense in a CI\r\nenvironment,
which is slower than a dev environment; 2) The timestamp to\r\nfilter
out requests was calculated after the task was triggered, and if\r\nthe
task is executed fast enough, it could lead to empty
responses\r\nbecause of
that.","sha":"358e104ebcc2bfcfeec3457b2e913d071d27745d"}}]}] BACKPORT-->

Co-authored-by: Sebastián Zaffarano <[email protected]>
@szaffarano szaffarano mentioned this issue Oct 1, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@szaffarano szaffarano

Labels
failed-es-promotion failed-test A test failure on a tracked branch, potentially flaky-test Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Telemetry][Security Solution] Fix flaky tests elastic/kibana
5 participants
@szaffarano @mistic @delanni @elasticmachine @kibanamachine