Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/uptime/certificates·ts - Uptime app with generated data certificates page "before each" hook for "performs search against monitor id" #67070

Closed
kibanamachine opened this issue May 20, 2020 · 6 comments · Fixed by #67167
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented May 20, 2020

A test failed on a tracked branch

Error: retry.try timeout: Error: expected testSubject(uptimeCertificatesPage) to exist
    at TestSubjects.existOrFail (/dev/shm/workspace/kibana/test/functional/services/common/test_subjects.ts:62:15)
    at onFailure (/dev/shm/workspace/kibana/test/common/services/retry/retry_for_success.ts:28:9)
    at retryForSuccess (/dev/shm/workspace/kibana/test/common/services/retry/retry_for_success.ts:68:13)

First failure: Jenkins Build

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label May 20, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-test-triage (failed-test)

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@spalger spalger added the Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability label May 20, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/uptime (Team:uptime)

@spalger
Copy link
Contributor

spalger commented May 20, 2020

This failed on master and 7.x pretty close to each other, hasn't failed on PRs yet. Looks like @andrewvc and @shahzad31 have modified these tests recently.

Screenshot during failure:
image

The test logs indicate that it was looking for the uptimeCertificatesPage element, but it kept retrying and never reclicked the uptimeCertificatesLink element, probably because the testSubjects.existOrFail() method uses a different default timeout... You might want to use config.get('timeouts.waitForExists') as the timeout for this call or just use the exists() method and throw your own error when it's missing.

@shahzad31
Copy link
Contributor

I will take a look at this, feel free to skip them if seems obstructive.

@spalger
Copy link
Contributor

spalger commented May 20, 2020

I assume the new flakiness was introduced by #66806, so in the last week or so they've only failed at one point in time, so I don't feel the need to skip them. Additionally, I suspect that PR makes things more stable than they were before so I don't feel the urge to revert it.

That said, I think the flakiness is coming from a shared service so it probably means that many tests are more likely to be flaky until we get this fixed, so being quick to fix would definitely be appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants