-
Notifications
You must be signed in to change notification settings - Fork 119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix an incorrect outcome (corner cases) #917
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pawelpasterz
requested review from
bootstraponline,
adamfilipow92,
piotradamczyk5 and
jan-goral
July 31, 2020 13:40
pawelpasterz
changed the title
Fix incorrect outcome (corner cases)
Fix an incorrect outcome (corner cases)
Jul 31, 2020
jan-goral
reviewed
Jul 31, 2020
} | ||
|
||
@Test | ||
fun `savedMatrix should have failed outcome when at least one test is failed and the last one is flaky`() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggested change
fun `savedMatrix should have failed outcome when at least one test is failed and the last one is flaky`() { | |
fun `savedMatrix should have failed outcome when at least one test is failed`() { |
pawelpasterz
force-pushed
the
fix-incorrect-outcome
branch
from
July 31, 2020 14:01
559904e
to
61c2dbf
Compare
piotradamczyk5
previously approved these changes
Aug 3, 2020
adamfilipow92
previously approved these changes
Aug 3, 2020
jan-goral
previously approved these changes
Aug 3, 2020
pawelpasterz
dismissed stale reviews from jan-goral, adamfilipow92, and piotradamczyk5
via
August 3, 2020 13:31
c9926e9
pawelpasterz
force-pushed
the
fix-incorrect-outcome
branch
from
August 3, 2020 13:36
c9926e9
to
7d733ed
Compare
adamfilipow92
previously approved these changes
Aug 3, 2020
pawelpasterz
force-pushed
the
fix-incorrect-outcome
branch
from
August 4, 2020 12:04
7d733ed
to
2a6e254
Compare
piotradamczyk5
previously approved these changes
Aug 4, 2020
pawelpasterz
force-pushed
the
fix-incorrect-outcome
branch
from
August 4, 2020 19:37
2a6e254
to
6cc5607
Compare
pawelpasterz
force-pushed
the
fix-incorrect-outcome
branch
from
August 5, 2020 11:36
6cc5607
to
f978d30
Compare
piotradamczyk5
approved these changes
Aug 5, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #914
Due to the incorrectly implemented logic regarding the status update, there were cases when flaky tests obscure failed results.
The following must occur:
failure
when failed test/shards/executions are processedfailure
but execution issuccess
it setsSavedMatrix
outcome to flakywhen
conditionfailure
check is never reachedProposed fix
failure
firstTest Plan
All tests pass and build finish successfully.
No easy way to verify it without previously prepared data/results
Checklist