Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect and report flaky tests #1049

Conversation

michalvavrik
Copy link
Member

@michalvavrik michalvavrik commented Feb 21, 2024

Summary

Flaky tests from SureFire and FailSafe reports are aggregated by a Maven extension added in this PR. The flaky report can be used in Jenkins or in GitHub Actions. Here on GitHub when PR contains flaky tests, we add triage/flaky-test label and comment which job contained flaky tests so that someone can inspect artifacts with actual test names.

Please check the relevant options

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Dependency update
  • Refactoring
  • Breaking change (fix or feature that would cause existing functionality to change)
  • This change requires a documentation update
  • This change requires execution against OCP (use run tests phrase in comment)

Checklist:

  • Example scenarios has been updated / added
  • Methods and classes used in PR scenarios are meaningful
  • Commits are well encapsulated and follow the best practices

@michalvavrik michalvavrik requested a review from mjurc February 21, 2024 23:10
@mjurc mjurc merged commit bffa1b1 into quarkus-qe:main Feb 22, 2024
8 checks passed
@michalvavrik michalvavrik deleted the feature/add-flaky-test-detection-and-reporting branch February 22, 2024 09:33
@gtroitsk gtroitsk mentioned this pull request Mar 21, 2024
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants