-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
(Accessibility) [Stack Monitoring] Failing/Flaky test "a11y tests for self monitoring home page" #163197
Comments
Pinging @elastic/kibana-qa (Team:QA) |
There is an update from ES team: https://elastic.slack.com/archives/C0D8ST60Y/p1691409068837599
Which seems to be connected directly to #162508 PR It passed as single test, because, before running in scope of all tests, state of the test environment changed during security solution tests execution, which created data stream. |
Issue described in comment above has been fixed by @andreidan team and the failing test from #162508 is passing now. So, I am closing this ticket |
Thanks @vitaliidm for keeping this issue updated. Really appreciated your willingness to make sure we got the RCA. Cheers! |
In #162508 one of Accessibility tests started to fail, which seemed to be not connected to changes in that PR
Failed test
Test itself
As it seems from test, =it supposed to wait 30s, before timing out, but fails with error
I ran it on flaky-test-runner, and it consistently failed:
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2779
But when I ran only this one failed test, not all in the file, it passed (on PR branch)
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2781
It passes on main
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/2780
Also this test passes when I run it locally, with the scope of single test file
x-pack/test/accessibility/apps/stack_monitoring.ts
on PR branchLocal test run logs
```It looks like, this test might depend on state of other tests, since it executes fine alone and in small scope of tests
Also, tests in
x-pack/test/accessibility/apps/stack_monitoring.ts
do not clear environment data. So second run, would have some state, created during previous run, leading to multiple failures - requiring fresh environmentThe text was updated successfully, but these errors were encountered: