Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report tests that passes on retry #5257

Closed
kiranparajuli589 opened this issue Jun 14, 2021 · 2 comments
Closed

Report tests that passes on retry #5257

kiranparajuli589 opened this issue Jun 14, 2021 · 2 comments
Assignees
Labels

Comments

@kiranparajuli589
Copy link
Contributor

Description

Acceptance tests are retried 1 time if it fails for the first time. If a test passes on a retry, the CI will go green silently. If not examined properly, these passes ( with a retry ) will go unnoticed.
We should be able to report (to GitHub or rocket chat 🤔 ) if tests pass on a retry so that they can be investigated at the same time when they arise, but not when they explode.

Motivation

  • catching/reporting flaky tests
@dpakach
Copy link
Contributor

dpakach commented Jun 24, 2021

The tests that pass on retry are now reported in github PR
eg. #5287 (comment)

@dpakach dpakach closed this as completed Jun 24, 2021
@phil-davis
Copy link
Contributor

Example: #5359 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants